Mulesoft

Externalising Mule 3.x Properties

Published on January 30, 2019

Externalising Mule 3.x Properties

Properties always make life easier. When it comes to Mule, there is a sweet and simple set of functionality to load properties based on the environment. Usually, a property file is used in the environment where the application is getting deployed, however, this involves creating multiple property files, one for each environment e.g. Dev, QA, SIT, Prod etc. A detailed reference to this approach can be found here.

Although this approach is straightforward to implement, it presents an issue in that all the configuration properties are embedded in the code. Anyone who has access to the code can access all your production systems which is a situation you never really want to get in to. In most organisations the IT team will never give away production credentials for systems.

CI/CD best practices recommend having a single deployable artifact that can run in all environments ( dev – prod ) by using different properties. To implement something like this properties will have to be fetched from an external source system like a simple file server, SFTP, Database or an API. This is exactly what we will be describing further in this post.

What to do?

We need to load properties from an external source (SFTP in this example) by overriding the Spring PropertyPlaceholderConfigurer class. Loading of properties will be done in the same way as Mule or Spring loads properties, with a bit of customisation.

How to do it?

There are three areas we will have to cover to get this working:

  • Adding dependencies in POM.xml
  • Writing custom java class for loading properties.
  • Creating a bean to trigger java class in the mule configuration xml file

Adding dependencies in POM.xml

To override Spring context properties we will need to load them first. These three dependencies are needed for Spring, SFTP and logging.

<!-- https://mvnrepository.com/artifact/com.jcraft/jsch -->
		<dependency>
		    <groupId>com.jcraft</groupId>
		    <artifactId>jsch</artifactId>
		    <version>0.1.55</version>
		</dependency>
<!-- https://mvnrepository.com/artifact/log4j/log4j -->
		<dependency>
			<groupId>log4j</groupId>
			<artifactId>log4j</artifactId>
			<version>1.2.5</version>
		</dependency>		
<!--https://mvnrepository.com/artifact/org.springframework/spring-beans -->
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-beans</artifactId>
			<version>4.1.9.RELEASE</version>
		</dependency>

Writing custom java class for loading properties

Create a class called ExternalProperties in src/main/java with the following code. This class reads a property file from a SFTP server but can be modified to pull properties from an API or Database as well. This class should also contain the decryption logic for all properties if properties are encrypted at source.

package com.integralzone.mule.properties;

import java.io.IOException;
import java.io.StringReader;
import java.util.Properties;

import com.jcraft.jsch.Channel;
import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import com.jcraft.jsch.SftpException;

import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;

import org.springframework.beans.factory.config.PropertyPlaceholderConfigurer;

import org.apache.log4j.Logger;

/**
 * @author Vibhanshu Thakur
 * Extending PropertyPlaceholderConfigurer which is Spring's way 
 * of loading properties into application.
 *
 */
public class ExternalProperties extends PropertyPlaceholderConfigurer {
	//for logging
	public static final Logger LOGGER = Logger.getLogger(ExternalProperties.class);
	
	/* (non-Javadoc)
	 * @see org.springframework.beans.factory.config.PropertyResourceConfigurer#convertProperties(java.util.Properties)
	 * This functions gets automatically triggered on creation of a bean of this class.
	 * Parameter props is not be passed at any point, function will self trigger.
	 */
	protected void convertProperties(Properties props) {
		LOGGER.info("properties loading process started");
		
		try {
			//loading properties from SFTP file into a StringReader(expected type for load())
			StringReader interimReaderProperties = new StringReader(ExternalProperties.readSFTP());
			//loading properties into application - thats how easy it is
			props.load(interimReaderProperties);
		} catch (IOException e1) {
			e1.printStackTrace();
		}
		
		LOGGER.info("Properties loaded successfully");
	}
	
	
	/**
     * Function to read asked file from SFTP
	 * @return String containing the entire file picked up from SFTP
	 * @throws IOException
	 */
	private static String readSFTP() throws IOException {
        JSch jsch = new JSch();
        Session session = null;
        try {
        	//sftp credentials and details
            session = jsch.getSession("user", "127.0.0.1", 22);
            session.setConfig("StrictHostKeyChecking", "no");
            session.setPassword("password");
            session.connect();

            Channel channel = session.openChannel("sftp");
            channel.connect();
            ChannelSftp sftpChannel = (ChannelSftp) channel;
            // file to be picked up
            InputStream stream = sftpChannel.get("config.properties");
            String finalFile = "";
            try {
            	//reading file
                BufferedReader br = new BufferedReader(new InputStreamReader(stream));
                String line;
                while ((line = br.readLine()) != null) {
                	//storing content of file to string for consolidating
                	finalFile+=line + "\n";
                }
            } catch (IOException io) {
                System.out.println("Exception occurred during reading file from SFTP server due to " + io.getMessage());
                io.getMessage();

            } catch (Exception e) {
                System.out.println("Exception occurred during reading file from SFTP server due to " + e.getMessage());
                e.getMessage();

            }
            sftpChannel.exit();
            session.disconnect();
            return finalFile;
        } catch (JSchException e) {
            e.printStackTrace();
            return null;
        } catch (SftpException e) {
            e.printStackTrace();
            return null;
        }
    }
}

Creating a bean to trigger the java class in the mule configuration xml file

Now the only step remaining is to use the code that we have written so far. For this, a bean should be created in any one of the mule configuration xml files in your application.

Make sure you remove any <context:property-placeholder> or <secure-property-placeholder:config> tags from all your xmls. Since we override the Spring property loader, these tags will not work and fail deployment of your application. We will also see in the bean definition how to load properties from a local/accessible file system (doing what context:property-placeholder does) so that the basic property loading functionality is not missed.

		<spring:beans>
		<spring:bean id="propertyLoader" class="com.integralzone.mule.properties.ExternalProperties">
			<spring:property name="locations">
				<spring:list>
					<spring:value>common.properties</spring:value>
					<spring:value>error_codes.properties</spring:value>
					<spring:value>logger_config.properties</spring:value>
				</spring:list>
			</spring:property>
		</spring:bean>
	</spring:beans>


The Spring bean created in the above code will automatically trigger our function to load properties from the SFTP location. Your code is now ready to read property files from external SFTP location.

If there is a requirement to load other property files present in src/main/resources or other local file directories then a property of the type list called “locations” should be passed while creating the propertyLoader bean. The value of this list should contain all the property file names in the required sequence to load these files like common, error_code & logger_config property in the above example. No additional code/setup is required to load these common properties.

Let us know if this was helpful.

This post was written by

7 Replies to “Externalising Mule 3.x Properties”

    1. Hi MuleDevil,
      Mule 4 actually blocks custom java code to go ahead and override PropertyPlaceholderConfigurer. Hence this method cannot be used. We will soon publish another blog on how to do this in Mule 4.
      Thanks for asking.

  1. I love this design and am trying to implement it but am running in to some errors. I can see in the log that it is trying to load the properties file and even references the custom ExternalProperties class but it comes back with:
    Could not load properties; nested exception is java.io.FileNotFoundException: class path resource [global.properties] cannot be opened because it does not exist

    when I debug, it does not hit my code which makes me think that it is trying to find a local file rather than firing my class. I’ve gone over your example code but must be missing something.

    also, are we able to use the credentialsvault to decrypt the encrypted entries?

  2. I’m not sure what I did different, but it’s working now!!! This is exactly what I needed.
    I added the following property to the bean but not sure it that was the issue or not:
    true

    Now, the next hurdle is decrypting by using the credentialsVault. Is there a way to deal with encrypted properties the way that the secure-property-placeholder works?

    1. Hi Dan,
      you can achieve what secure property placeholder does by using javax.crypto jar. Here are some points to consider when you do it:
      – You will need to first read all property values and then decrypt them one by one(like a loop) as not all properties are encrypted.
      – Make sure you do not try to decryt a property which wasn’t encrypted in the first place. All encrypted values would be inside ![encrypted_value]. You can use string matching to recognize which ones to decrypt and then load them.
      – You need to make sure that all encrypted property files that you are trying to load have the same encryption algorithm. If not, ideal solution would be to have an http call before loading the properties from SFTP or get properties via an http call and the http response can contain encryption algorithm and other details.
      – Make sure not to hardcode the encryption key as people will be able to see it in your scm(github or bitbucket). Also make sure that encryption key, encryption algorithm and the properties are coming from different sources.

      Let me know if i have answered your query. As always, thank you for asking these questions.

      Cheers
      Vibhanshu

Leave a Reply

Your email address will not be published. Required fields are marked *

Other Blog Posts

Other Blog Posts

Choosing Worker Size in CloudHub

Running applications in Mulesoft’s CloudHub is probably the easiest way for any organization using Mule to manage and deploy applications. It guarantees; 99.99% up-time per annum, zero downtime during application deployment, easy management of properties, flow monitoring, application insights, persistent queues, schedulers tracking, persistent object stores, and the list goes on. CloudHub is a component …

Read more