mongoDB replica set and partitioned cluster

mongoDb replica set

It mainly provides two functions

1. When data is written to the Primary node, copy the data to another replica node

2. When the primary node fails, a new replacement node is automatically selected

Typical replica set structure

A typical replication set consists of three or more nodes with voting rights. One Primary node receives write operations, read operations and voting during election, and two or more secondary nodes replicate new data on the Primary node and voting during election

How is data replicated

When a modification operation, whether insert, update or delete, reaches the master node, its operation on the data will be recorded (after some necessary transformations). These records are called oplog

The slave node continuously obtains the oplog of the new master node from the master node and plays it back on its own data, so as to keep consistent with the data of the master node.

It is very similar to the AOF method in redis persistence.

Replication set setup

  1. Create data directory file

mkdir -p /data/db{1,2,3}


2. Prepare the configuration file of each database (each mongod process of the replication set should be located on a different server. We now run three instances on one server, so we need to configure them respectively)

1.Different ports: 280172801828019.
2.Different data directories
3. Different log file paths. Used in instances
/data/db3/mongod.log   touch mongod.log

Then edit the mongod.conf file  


  destination: file
  path: /data/db1/mongod.log
  logAppend: true
  dbPath: /data/db1
  port: 28017
  replSetName: rs0
  fork: true


  destination: file
  path: /data/db2/mongod.log
  logAppend: true
  dbPath: /data/db2
  port: 28018
  replSetName: rs0
  fork: true


  destination: file
  path: /data/db3/mongod.log
  logAppend: true
  dbPath: /data/db3
  port: 28019
  replSetName: rs0
  fork: true

  Start separately

mongod -f /data/db1/mongod.conf

mongod -f /data/db2/mongod.conf

mongod -f /data/db3/mongod.conf

Link the first mongo --port 28017

It is found that the three machines are still independent of each other. At this time, configure them to form a cluster



View the replica set status through rs.status()


  It is found that the port 28017 is the master node and the other two are the slave nodes.

By default, non primary nodes are not allowed to read data.

You can turn on the read permission by executing rs.secondaryOk()


MongoDB partitioned cluster

Composition of fragment cluster

  Build a cluster of 2 slices

1. Create a data directory: it is prepared for two replica sets. Each replica set has three instances and a total of 6 data nodes.  

mkdir -p /data/shard1    /data/shard1second1     /data/shard1second2     /data/shard2    /data/shard2second1     /data/shard2second2   

2. Create 6 log files in total

touch  /data/shard1/mongod.log     /data/shard1second1/mongod.log    /data/shard1second2/mongod.log    /data/shard2/mongod.log     /data/shard2second1/mongod.log    /data/shard2second2/mongod.log 

  3. Start the first mongod sharding instance (three instances in total)

mongod --bind_ip --replSet shard1 --dbpath /data/shard1 --logpath /data/shard1/mongod.log --port 27010   --fork   --shardsvr

mongod --bind_ip --replSet shard1 --dbpath /data/shard1second1  --logpath /data/shard1second1/mongod.log --port 27011   --fork   --shardsvr
mongod --bind_ip --replSet shard1 --dbpath /data/shard1second2 --logpath /data/shard1second2/mongod.log --port 27012   --fork   --shardsvr

4. After the mongod instances of the first shard are started, add them to the replication set


Configure Config replication set: there are three instances in total

1. Create data directory:

mkdir -p /data/config    /data/configsecond1     /data/configsecond2

  2. Create log file

touch  /data/config/mongod.log     /data/configsecond1/mongod.log    /data/configsecond2/mongod.log 

3. Start the configuration replication set

mongod --bind_ip --replSet config --dbpath /data/config  --logpath /data/config/mongod.log --port 37010   --fork   --configsvr 
mongod --bind_ip --replSet config --dbpath /data/configsecond1 --logpath /data/configsecond1/mongod.log --port 37011   --fork   --configsvr 
mongod --bind_ip --replSet config --dbpath /data/configsecond2  --logpath /data/configsecond2/mongod.log --port 37012   --fork   --configsvr 

4. Configure the replica set for initialization


Configure mongs routing node

1. To start the mongos instance, you need to specify the address list of the configuration server

mongos --bind_ip --logpath /data/mongos/mongos.log --port 4000 --fork --configdb config/,,

  Where configdb is the address list of the configuration server

2. Connect to mongos and add Shards

mongo --port 4000 local direct connection

Execute script


View slice status


To create a shard table:

  1. MongoDB sharding is set based. Even if there is a sharding cluster, the data will be automatically sharded, and the sharding table needs to be displayed

First, you need to enable database fragmentation

sh.enableSharding("library name");

For example:


sh.shardCollection("library name. Collection name", {_id: "hashed"});

sh.shardCollection("order.account",{_id: "hashed"});

Dynamic capacity expansion

  Create a second replica set to implement sharding

mongod --bind_ip --replSet shard2 --dbpath /data/shard2 --logpath /data/shard2/mongod.log --port 27013   --fork   --shardsvr

mongod --bind_ip --replSet shard2 --dbpath /data/shard2second1  --logpath /data/shard2second1/mongod.log --port 27014   --fork   --shardsvr
mongod --bind_ip --replSet shard2 --dbpath /data/shard2second2 --logpath /data/shard2second2/mongod.log --port 27015   --fork   --shardsvr

Initialize replication set


Join cluster partition

mongo --port 4000

MongoDB application practice  

1. java Native Client

Introducing maven

public class QuickStart {

    public static void main(String[] args) {
        // Mongod connecting to the local default port
        // MongoClient mongoClient = MongoClients.create()
        // Mongod that connects to the specified port of the remote service
        // MongoClient mongoClient = MongoClients.create("mongodb://host1:27017");
        // Connect to the specified port replication set
        // MongoClient mongoClient = MongoClients.create("mongodb://host1:27017,host2:27017,host3:27017/?replicaSet=myReplicaSet");
        // Connect mongos route: connect one
        // MongoClient mongoClient = MongoClients.create( "mongodb://localhost:27017" );
        // Connecting multiple mongos routes
        MongoClient mongoClient = MongoClients.create("mongodb://");

        //Get database
        MongoDatabase database = mongoClient.getDatabase("productdb");

        // Get collection
        MongoCollection<Document> productdesc=database.getCollection( "productdesc" );

        Document doc = new Document("name", "MongoDB")
                .append("type", "database")
                .append("count", 1)
                .append("versions", Arrays.asList("v3.2", "v3.0", "v2.6"))
                .append("info", new Document("x", 203).append("y", 102));


        Bson eq = eq("name", "MongoDB");
        FindIterable<Document> find = productdesc.find(eq);
        Document first=find.first();




Spring Boot integration:

1. Introduce maven

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="" xmlns:xsi=""
      <relativePath/> <!-- lookup parent from repository -->
   <description>Demo project for Spring Boot</description>








  2. Add configuration class

For example, connect to mongos above and use MongoTemplate for database operation.

package com.example.demo.config;

import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoClients;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

public class AppConfig {

    public @Bean
    MongoClient mongoClient() {
        return MongoClients.create("mongodb://");

    public @Bean
    MongoTemplate mongoTemplate() {
        return new MongoTemplate(mongoClient(), "productdb");


3. Testing

package com.example.demo;

public class Person {

  private String id;
  private String name;
  private int age;

  public Person(String name, int age) { = name;
    this.age = age;

  public String getId() {
    return id;
  public String getName() {
    return name;
  public int getAge() {
    return age;

  public String toString() {
    return "Person [id=" + id + ", name=" + name + ", age=" + age + "]";


package com.example.demo;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.stereotype.Component;

import java.util.List;

import static;
import static;
import static;
public class ApplicationRunnerTest implements ApplicationRunner{

    private MongoTemplate mongoOps;

    public void run(ApplicationArguments applicationArguments) throws Exception {

        Person p = new Person("Joe", 34);

          // Insert document
        mongoOps.insert(p);"Insert: " + p);

        // consult your documentation
        p = mongoOps.findById(p.getId(), Person.class);"Found: " + p);

        // Update document
        mongoOps.updateFirst(query(where("name").is("Joe")), update("age", 35), Person.class);
        p = mongoOps.findOne(query(where("name").is("Joe")), Person.class);"Updated: " + p);

        // remove document

        // Check that deletion worked
        List<Person> people =  mongoOps.findAll(Person.class);"Number of people = : " + people.size());







Tags: MongoDB nosql

Posted on Wed, 01 Dec 2021 06:28:12 -0500 by skippy111