Record the whole process of SpringBoot multi data source configuration

  • 2021-12-11 07:19:12
  • OfStack

Preface to the table of contents Configuration file Dependency Build AbstractRoutingDataSource Data source switching Directory Summarize

Preface

The core of multiple data sources is injecting AbstractRoutingDataSource into the IOC container and how to switch data sources. The injection mode can be registered BeanDefinition or built Bean, and the switching mode of data source can be method parameter or annotation switching (others are not imagined), which is determined by requirements.

My requirement is to count the data of multiple databases and write the results into another database. The number of statistical databases is uncertain, which cannot be directly injected through @ Bean. It is also a statistical task, and the annotation switching of DAO layer cannot be met, so I choose to register (AbstractRoutingDataSource) BeanDefinition and switch method parameters. Let's take the statistics of Japanese and Korean users to the result database as an example.

Configuration file

master is the result library, and others are the statistical databases (china, japan can be identified by enumeration only 1, of course, String can also be used):


dynamic:
  dataSources:
    master:
      driver-class-name: com.mysql.cj.jdbc.Driver
      url: jdbc:mysql://localhost:3306/result?useUnicode=true&characterEncoding=utf8xxxxxxxx
      username: root
      password: 123456
    china:
      driver-class-name: com.mysql.cj.jdbc.Driver
      url: jdbc:mysql://localhost:3306/china?useUnicode=true&characterEncoding=utf8xxxxxxxx
      username: root
      password: 123456
    japan:
      driver-class-name: com.mysql.cj.jdbc.Driver
      url: jdbc:mysql://localhost:3306/japan?useUnicode=true&characterEncoding=utf8xxxxxxxx
      username: root
      password: 123456
    korea:
      driver-class-name: com.mysql.cj.jdbc.Driver
      url: jdbc:mysql://localhost:3306/korea?useUnicode=true&characterEncoding=utf8xxxxxxxx
      username: root
      password: 123456

Corresponding configuration class:


package com.statistics.dynamicds.core.config;

import com.statistics.dynamicds.core.Country;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;

import java.util.Map;

import static com.statistics.dynamicds.core.config.MultiDataSourceProperties.PREFIX;

@Data
@Configuration
@ConfigurationProperties(prefix = PREFIX)
public class MultiDataSourceProperties {
  public static final String PREFIX = "dynamic";
  private Map<Country, DataSourceProperties> dataSources;

  @Data
  public static class DataSourceProperties {
    private String driverClassName;
    private String url;
    private String username;
    private String password;
  }
}
package com.statistics.dynamicds.core;

public enum Country {
  MASTER("master", 0),

  CHINA("china", 86),
  JAPAN("japan", 81),
  KOREA("korea", 82),
  //  Omitted by other countries 

  private final String name;
  private final int id;

  Country(String name, int id) {
    this.name = name;
    this.id = id;
  }

  public int getId() {
    return id;
  }

  public String getName() {
    return name;
  }
}

Dependency

JPA for ORM, SpringBoot version 2.3. 7. RELEASE, simplifying GetSet through Lombok.


<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>

<dependency>
     <groupId>org.projectlombok</groupId>
     <artifactId>lombok</artifactId>
     <version>1.18.22</version>
     <scope>provided</scope>
</dependency>

Constructing AbstractRoutingDataSource

The dynamic data source of Spring needs to be injected into AbstractRoutingDataSource, because the statistical data source in the configuration file is not fixed, so it cannot be injected through @ Bean annotation, and needs to be built manually.

Add @ Import (MultiDataSourceImportBeanDefinitionRegistrar. class) to the startup class.

Add @ Import (MultiDataSourceImportBeanDefinitionRegistrar. class) to the startup class.

To add @ Import (MultiDataSourceImportBeanDefinitionRegistrar. class) to the startup class, write 3 lines for the important thing.


package com.statistics.dynamicds.autoconfig;

import com.statistics.dynamicds.core.DynamicDataSourceRouter;
import com.statistics.dynamicds.core.Country;
import com.statistics.dynamicds.core.config.MultiDataSourceProperties;
import com.zaxxer.hikari.HikariDataSource;
import org.springframework.beans.factory.support.AbstractBeanDefinition;
import org.springframework.beans.factory.support.BeanDefinitionBuilder;
import org.springframework.beans.factory.support.BeanDefinitionRegistry;
import org.springframework.boot.context.properties.bind.Binder;
import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.context.EnvironmentAware;
import org.springframework.context.annotation.ImportBeanDefinitionRegistrar;
import org.springframework.core.env.Environment;
import org.springframework.core.type.AnnotationMetadata;

import javax.annotation.Nonnull;
import java.util.Map;
import java.util.stream.Collectors;

import static com.statistics.dynamicds.core.config.MultiDataSourceProperties.PREFIX;

public class MultiDataSourceImportBeanDefinitionRegistrar implements ImportBeanDefinitionRegistrar, EnvironmentAware {
  public static final String DATASOURCE_BEANNAME = "dynamicDataSourceRouter";
  private Environment environment;

  @Override
  public void registerBeanDefinitions(@Nonnull AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry registry) {
    MultiDataSourceProperties multiDataSourceProperties = Binder.get(environment)
            .bind(PREFIX, MultiDataSourceProperties.class)
            .orElseThrow(() -> new RuntimeException("no found dynamicds config"));
    final HikariDataSource[] defaultTargetDataSource = {null};
    Map<Country, HikariDataSource> targetDataSources = multiDataSourceProperties.getDataSources().entrySet().stream()
            .collect(Collectors.toMap(
                    Map.Entry::getKey,
                    entry -> {
              MultiDataSourceProperties.DataSourceProperties dataSourceProperties = entry.getValue();
              HikariDataSource dataSource = DataSourceBuilder.create()
                      .type(HikariDataSource.class)
                      .driverClassName(dataSourceProperties.getDriverClassName())
                      .url(dataSourceProperties.getUrl())
                      .username(dataSourceProperties.getUsername())
                      .password(dataSourceProperties.getPassword())
                      .build();
              dataSource.setPoolName("HikariPool-" + entry.getKey());
              if (Country.MASTER == entry.getKey()) {
                defaultTargetDataSource[0] = dataSource;
              }
              return dataSource;
            }));
    AbstractBeanDefinition beanDefinition = BeanDefinitionBuilder.genericBeanDefinition(DynamicDataSourceRouter.class)
            .addConstructorArgValue(defaultTargetDataSource[0])
            .addConstructorArgValue(targetDataSources)
            .getBeanDefinition();
    registry.registerBeanDefinition(DATASOURCE_BEANNAME, beanDefinition);
  }

  @Override
  public void setEnvironment(@Nonnull Environment environment) {
    this.environment = environment;
  }
}

In the above code, MultiDataSourceProperties was not obtained by @ Resource or @ Autowired because ImportBeanDefinitionRegistrar was executed very early, and the configuration parameter class of @ ConfigurationProperties has not been injected at this time, so it should be obtained manually (the annotation of @ ConfigurationProperties is added to enable other Bean in IOC container to obtain configured Country, so as to switch data sources).

The following is the implementation class DynamicDataSourceRouter for AbstractRoutingDataSource:


package com.statistics.dynamicds.core;

import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;

import java.util.Map;

public class DynamicDataSourceRouter extends AbstractRoutingDataSource {
  public DynamicDataSourceRouter(Object defaultTargetDataSource, Map<Object, Object> targetDataSources) {
    this.setDefaultTargetDataSource(defaultTargetDataSource);
    this.setTargetDataSources(targetDataSources);
  }

  @Override
  protected Object determineCurrentLookupKey() {
    return DataSourceContextHolder.getLookupKey();
  }
}

Data source switching

Switching of data sources is controlled by DataSourceContextHolder and section DynamicDataSourceAspect:


package com.statistics.dynamicds.core;

public class DataSourceContextHolder {
  private static final ThreadLocal<Country> HOLDER = ThreadLocal.withInitial(() -> Country.MASTER);

  public static void setLookupKey(Country lookUpKey) {
    HOLDER.set(lookUpKey);
  }

  public static Country getLookupKey() {
    return HOLDER.get();
  }

  public static void clear() {
    HOLDER.remove();
  }
}
package com.statistics.dynamicds.core;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.springframework.stereotype.Component;

@Aspect
@Component
public class DynamicDataSourceAspect {

  @Pointcut("execution(* com.statistics.dao..*.*(..))")
  void aspect() {

  }

  @Around("aspect()")
  public Object around(ProceedingJoinPoint joinPoint) throws Throwable {
    for (Object arg : joinPoint.getArgs()) {
      if (arg instanceof Country) {
        DataSourceContextHolder.setLookupKey((Country) arg);
        break;
      }
    }
    try {
      return joinPoint.proceed();
    }finally {
      DataSourceContextHolder.clear();
    }
  }
}

Directory

.
com
statistics
StatisticsApplication. java
The
(ES 117EN)
UserDao. java
The
-dynamicds
→ ES 125EN
MultiDataSourceImportBeanDefinitionRegistrar. java
The
-core
DataSourceContextHolder. java
DynamicDataSourceAspect. java
DynamicDataSourceRouter. java
Province. java
The
-config
MultiDataSourceProperties. java

Summarize

The multi-data source configuration is completed above, and it is only necessary to add an Country enumeration to the method parameters of dao layer when using it.

If you can't use enumeration to identify the data source, you can also replace it with String. Other information about this data source can be added to an map in the internal class DataSourceProperties. In short, it is expanded according to your own needs.


Related articles: