Mongod

• You can configure mongodb in ndsk.config.js
• Note: The current node driver version for mongodb is 6.10 documentation

 module.exports = (isPro)=>{
     return {
         server:{
             plugins:{
                 mongodb:{
                     name:"mongo",  // Plug-in name
                     client:[
                         {
                             name:'client',    //request.mongo.client  or  server.mongo.client
                             uri:'mongodb+srv://<db_username>:<db_password>@cluster0.rw45myx.mongodb.net/?retryWrites=true&w=majority&appName=Cluster0', 
                             config:{}      //mongodb config
                         },
                         // {
                         //    name:'client1',   //request.mongo.client1  or server.mongo.client1
                         //    uri:'mongodb+srv://********',
                         //    config:{}     //mongodb config
                         //}
                     ]
                 }
             }
         }
     }
 }

server.plugins.mongodb.name

• A string called with request.mongo.client or server.mongo.client if you set mongo, or request.test.client or server.test.client if you set test.

server.plugins.mongodb.client

• An array that allows multiple mongodb links to be added.

• Note: Your mongodb version must be >= v5.0

• After successful configuration you can use request.mongo.client in the route or server.mongo.client in the plugin for database operations.

Translated with DeepL.com (free version)

export default async (request)=>{
    const mongo = request.mongo.client;
    return await mongo.db('ndsk').collection('test').findOne()
}

exports.plugin = {
    name: 'mongoTest',
    register: (server) =>{
        server.events.on('start', async () => {
            const mongo = server.mongo.client;
            const data = await mongo.db('ndsk').collection('test').findOne();
            console.log(data)
        });
    }
}

Field Validation

• Although mongodb comes with field validation, it may not be a good idea for us to use the framework’s own validation, so here’s how to implement it.

my-nodestack-app
├── src
│   ├── routes
│   │   │── mongodb
│   │   │   │── action.js
│   ├── mongodb
│   │   │── client
│   │   │   │── ndsk
│   │   │   │   │── test.js
├── ndsk.config.js
└── package.json

const Joi = require('joi');

//insert
export const insert = Joi.array().items({
    username: Joi.string().default('rocky'),
    age:Joi.number().min(0).error(new Error('age must be a number')).required(),
}).required().single();

//update
export const update =  Joi.object().keys({
    age:Joi.number().min(0).required()
}).required();

const Joi = require('joi');
export const options = {
    validate:{
        query:Joi.object({
            action:Joi.string().valid('insertOne','updateOne').error(new Error('The value of the action parameter must be updateOne or insertOne')).required(),
            age:Joi.any().required()
        }),
    }
}
export default async (request)=>{
    const collection = request.mongo.client.db('ndsk').collection('test');
    const {action,age} = request.query;
    if(action === 'insertOne'){
        await collection.insertOne({age}); 
    }else{
        await collection.updateOne({username:'rocky'},{$set:{age}});
    }
    return await collection.findOne({username:'rocky'}); 
}

• We create src/mongodb/client/ndsk/test.js file for validation, where client corresponds to the name of the data client you configured in ndsk.config.js, ndsk is the name of the database, and test is the name of the collection to be validated.

• The above validation means that the test collection will be manipulated in the client connection to the ndsk database, and the age field must be a number and must be zero.

• Visit http://localhost:3000/mongo/example?action=insertOne&age=30

• Visit http://localhost:3000/mongo/example?action=insertOne&age=test

• Visit http://localhost:3000/mongo/example?action=updateOne&age=20

• Visit http://localhost:3000/mongo/example?action=updateOne&age=test

• mongodb uses Joi for field validation, you can see documentation on Joi here

Note:

• insert should use Joi.array().items({...}) .required().single() for validation, as you may be inserting data in bulk
• update should be checked with Joi.object().keys({}).required()

Database management

• Use request.mongo.client.admin or server.mongo.client.admin to manage the database, Note: client is the name of the database you are configuring.
• You can also use primaryMongo.admin to get the first client

  const admin = request.mongo.client.admin; // in the plugin you should use server.mongo.client.admin
  const admin = primaryMongo.admin; // In the plugin you should use server.mongo.client.admin.

admin supports the following methods

createDatabase(db,collection)

• Creates a database with two required parameters, db: database name, collection: collection name, for example:

export default async (request,h)=>{
    const admin = request.mongo.client.admin
    await admin.createDatabase('demo','test');   //create demo database and add test collection
    return await admin.listDatabases();     
}

listDatabases

• List available databases

export default async (request)=>{
    return await request.primaryMongo.admin.listDatabases();  
}

buildInfo

• Retrieve server information for the current instance of the database client

export default async (request)=>{
    return await request.primaryMongo.admin.buildInfo();  
}

serverInfo

• Retrieve server information for the current instance of the database client

export default async (request)=>{
    return await request.primaryMongo.admin.serverInfo();  
}

serverStatus

• Server Status

export default async (request)=>{
    return await request.primaryMongo.admin.serverStatus();  
}

<! – #### addUser

• Add a user to the database

removeUser

• Remove a user from the database -->

replSetGetStatus

• Get ReplicaSet Status

export default async (request)=>{
    return await request.primaryMongo.admin.replSetGetStatus();  
}

ping

• Ping MongoDB server and retrieve results

export default async (request)=>{
    return await request.primaryMongo.admin.ping();  
}

withSession

• Portable Affairs

export default async (request)=>{
    const client = request.primaryMongo;
    let status = null;
    try {
        const db = client.db("testdb"); // 替换为你的数据库名称
        const collection1 = db.collection("test1"); // 替换为你的集合名称
        const collection2 = db.collection("test2"); // 替换为你的集合名称

        // 使用便携式事务
        await client.withSession(async (session) => {
            await session.withTransaction(async () => {
                // Execute transaction operations
                await collection1.insertOne({ name: "Alice", balance: 100 }, { session });
                await collection2.insertOne({ name: "Bob", balance: 200 }, { session });

                console.log("Transaction commit successful");
            });
        });
    } catch (error) {
        console.error("Transaction Failure:", error);
    } finally {
        await client.close();
    }
    return status
}

startSession

• Transaction Core API

export default async (request)=>{
    const client = request.primaryMongo;
    const session = client.startSession();
    let status = null;
    try {
        // Start transaction
        session.startTransaction();

        const db = client.db("demo"); // Replace with your database name
        const collection1 = db.collection("test1"); // Replace with your collection name
        const collection2 = db.collection("test2"); // Replace with your collection name

         // Execute the operation
        await collection1.insertOne({ name: "Alice", balance: 100 }, { session });
        await collection2.insertOne({ name: "Bob", balance: 200 }, { session });

        // Commit the transaction
        await session.commitTransaction();
        console.log("Transaction commit successful");
        status = 'success'
    } catch (error) {
        // Rollback transaction
        await session.abortTransaction();
        console.error("Transaction Rollback:", error);
        status = 'error'
    } finally {
        // End the session
        session.endSession();
        await client.close();
    }
    return status
}

watch

• Monitoring Changes in MongoDB

exports.plugin = {
    name: 'mongoWatch',
    register: (server) =>{
        server.events.on('start', async () => {
            // Listen for changes in the current collection
            const changeStream = await server.mongo.client.watch();
            changeStream.on("change", function (change) {
                console.log(change);
            });
        });
    }
}

Manipulate the database

• Use request.mongo.client.db(xxx) or server.mongo.client.db(xxx) to fetch the data, Note: client is the name you set in the configuration database, and xxx is the database name.

    const db = request.mongo.client.db('demo');

db supports the following methods of operation

createCollection

• Creating Collections

export default async (request)=>{
    return request.primaryMongo.db('demo').createCollection('test1')
}

renameCollection

• Renaming Collections

export default async (request)=>{
    return request.primaryMongo.db('demo').renameCollection('test2','test2-1')
}

dropCollection

• Delete collection

export default async (request)=>{
    return request.primaryMongo.db('demo').dropCollection('test1')
}

dropDatabase

• Deletion of databases

export default async (request)=>{
    return request.primaryMongo.db('testdb').dropDatabase()
}

listCollections

• List all collections in the database

export default async (request)=>{
    return request.primaryMongo.db('demo').listCollections()
}

stats

• Get all database statistics

export default async (request)=>{
    return request.primaryMongo.db('demo').stats()
}

command

• Run command

export default async (request)=>{
    return await request.primaryMongo.db('admin').command({dbStats: 1});  
}

Collection operations

• 使用 request.mongo.client.db('demo').collection('test') 获取demo数据库中test集合

    const collection = request.mongo.client.db('demo').collection('test');

collection supports the following operations

find

• Batch Inquiry

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').find().toArray()
}

• If you need to use automatic caching, use cacheArray.

// query name:'test' and cache it for 5s.
export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').find({name:'test'}).cacheArray(5000)
}

aggregate

• Aggregation operations (the framework automatically optimizes performance in aggregation operations)

export default async (request)=>{
    const personColl = request.primaryMongo.db('demo').collection('persons');
    return await personColl.aggregate([
    {$project:{_id:0,address:0 }},
    {$sort: {"dateofbirth": -1,}},
    {$limit: 3}
  ]).toArray();   // Use automatic caching, use cacheArray(), cf. find
}

export default async (request)=>{
    const personColl = request.primaryMongo.db('demo').collection('persons');
    const personData = [
    {
      person_id: "6392529400",
      firstname: "Elise",
      lastname: "Smith",
      dateofbirth: new Date("1972-01-13T09:32:07Z"),
      vocation: "ENGINEER",
      address: {
        number: 5625,
        street: "Tipa Circle",
        city: "Wojzinmoj",
      },
    },
    {
      person_id: "1723338115",
      firstname: "Olive",
      lastname: "Ranieri",
      dateofbirth: new Date("1985-05-12T23:14:30Z"),
      gender: "FEMALE",
      vocation: "ENGINEER",
      address: {
        number: 9303,
        street: "Mele Circle",
        city: "Tobihbo",
      },
    },
    {
      person_id: "8732762874",
      firstname: "Toni",
      lastname: "Jones",
      dateofbirth: new Date("1991-11-23T16:53:56Z"),
      vocation: "POLITICIAN",
      address: {
        number: 1,
        street: "High Street",
        city: "Upper Abbeywoodington",
      },
    },
    {
      person_id: "7363629563",
      firstname: "Bert",
      lastname: "Gooding",
      dateofbirth: new Date("1941-04-07T22:11:52Z"),
      vocation: "FLORIST",
      address: {
        number: 13,
        street: "Upper Bold Road",
        city: "Redringtonville",
      },
    },
    {
      person_id: "1029648329",
      firstname: "Sophie",
      lastname: "Celements",
      dateofbirth: new Date("1959-07-06T17:35:45Z"),
      vocation: "ENGINEER",
      address: {
        number: 5,
        street: "Innings Close",
        city: "Basilbridge",
      },
    },
    {
      person_id: "7363626383",
      firstname: "Carl",
      lastname: "Simmons",
      dateofbirth: new Date("1998-12-26T13:13:55Z"),
      vocation: "ENGINEER",
      address: {
        number: 187,
        street: "Hillside Road",
        city: "Kenningford",
      },
    },
    ];
    return await personColl.insertMany(personData);
}

insertOne

• Single insertion

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').insertOne({
        username:'admin',
        password:'123456'
    })
}

insertMany

• Batch Insertion

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').insertOne([
        {
            username:'admin',
            password:'123456'
        },
        {
            username:'admin1',
            password:'123456'
        }
    ])
}

findOne

• Find a single

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').findOne()
}

• Using automatic caching

// look for {name:'test'} and automatically cache it for 5s
export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').findOne({name:'test'},{cache:5000})
}

updateOne

• Update single, Note When upsert:true, to prevent multiple inserts during concurrency, a unique index is automatically added based on the query conditions.

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').updateOne(
        {username:"admin"},
        {$set:{password:11111}}
    )
}

updateMany

• Bulk update, Note Ditto

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').updateMany(
        {username:"admin"},
        {$set:{password:123123}}
    )
}

replaceOne

• Replacement of documents, Note Ditto

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').replaceOne(
        {username:"admin"},
        {username:'aaaa',password:123123}
    )
}

findOneAndUpdate

• Find and update individual documents, Note Ditto

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').findOneAndUpdate(
        {username:"aaaa"},
        {$set:{password:11111}}
    )
}

findOneAndReplace

• Find and replace individual documents, Note Ditto

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').findOneAndReplace(
        {username:"aaaa"},
        {password:2222}
    )
}

deleteOne

• Delete the current document

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').deleteOne(
        {password:2222}
    )
}

deleteMany

• Batch Deletion of Documents

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').deleteMany(
        {password:2222}
    )
}

findOneAndDelete

• Finding and Deleting When Documents

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').findOneAndDelete(
        {username:'admin'}
    )
}

count

• Get the number of documents matching the filter

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').count()
}

• Setting up automatic caching

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('article').count(
        {userId:"67360df06f727702930f1254"},
        {cache:5000}   //自动缓存5s
    );
}

countDocuments

• Same as above, if you want to set up automatic caching.

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').countDocuments()
}

estimatedDocumentCount

• Same as above, automatic caching cannot be set for the current method

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').estimatedDocumentCount()
}

distinct

• Returns a list of distinct values for a given key in a collection

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').distinct('username')
}

createIndex

• Creating Indexes

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').createIndex({
        "username": 1
    })
}

createIndexes

• Batch Index Creation

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').createIndexes([
        {"username": 1},
        {"password": 1}
    ])

    //
    const {inprog} = await request.mongo.client.db('admin').command({ "currentOp": 1, "active": true ,"command.createIndexes": { $exists : true }});
    const op = inprog.find(e=>e.msg);
    return op ? op.msg : "create success"
}

Get index creation progress

export default async (request)=>{
    const {inprog} = await request.mongo.client.db('admin').command({ "currentOp": 1, "active": true ,"command.createIndexes": { $exists : true }});
    const opt = inprog.find(e=>e.msg && e.command && e.command.createIndexes === 'article');
    return opt ? opt : 'successed'
}

indexInformation

• Retrieves index information for this collection.

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').indexInformation()
}

dropIndex

• Delete Index

export default async (request)=>{
    return request.primaryMongo.db('demo').collection('test').dropIndex('username_1')
}

dropIndexes

• Batch Deletion of Indexes

export default async (request)=>{
    const indexs = [{"username": 1 }, {"password": 1}];
    await request.primaryMongo.db('demo').collection('test').createIndexes(indexs)
    return await request.primaryMongo.db('demo').collection('test').dropIndexes(indexs)
}

watch

• Monitor the current set operation

exports.plugin = {
    name: 'mongoWatch',
    register: (server) =>{
        server.events.on('start', async () => {
            //Listen for changes in the current collection, //Monitor the current collection, //Monitor the current collection, //Monitor the current collection.
            const changeStream = await server.mongo.client.db('demo').collection('test').watch();
            changeStream.on("change", function (change) {
                console.log(change);
            });
        });
    }
}

sample

• Randomly obtaining a specified amount of data is used as follows

return await user.sample(
        {
            size:10,                           //Required, randomized quantity
            project:{_id:1,name:1}             //Optional, filter fields
        },
        {
            limit:100000,                      //Optional, default: 100000, maximum number of queries
            maxTimeMS:500,                     //Optional, default: 500, query timeout
            cache:5000,                        //Results Cache 5s
        }
)

pagination

• A custom method for paging queries that automatically optimizes the performance of aggregate, making it possible to query tens of millions or more of data without any stress, as follows

export default async (request)=>{
    const collection = request.mongo.client.db('ndsk').collection('test');
    return await collection.pagination({
        cache:5000,                     //whether to auto-cache, 5000 means cache 5s
        query:{},                       //Query condition, an object Default: none, in big data paging query be sure to add filter conditions, already indexed, otherwise it will cause slow queries. For example _id:{$gt:'.....'}
        sort:{id:-1},                   //Sort, an object, default None
        page:1,                         //Current page, default 1
        limit:10,                       //Number of displays per page, default None
        count:true,                     //whether to count the total number, boolean or an object, if: true value is: { limit:10000000, maxTimeMS:500}, large data will cause the query is very slow, if you want to count the details of the data it is recommended to set to false, and then a separate interface request, or set the query cache
        // project:{},                  //Field filter, default None
        // lookup:[],                   //Aggregate association, an array, default None
        // group:{}                     //Aggregate association, an array, default None
    },{maxTimeMS:500});
}

Examples

• Query ten of the ten million pieces of data and manage the rest of the set.

export default async (request)=>{
    const db = request.mongo.client.db('demo')
    const article = db.collection('article');

    //Get the progress of index creation
    const getCreateIndexProgress = async ()=>{
        const {inprog} = await request.mongo.client.db('admin').command({ "currentOp": 1, "active": true ,"command.createIndexes": { $exists : true }});
        return inprog.find(e=>e.msg && e.command && e.command.createIndexes === 'article');
    }
    const op = await getCreateIndexProgress();
    if(op){
        return op.msg
    }
    return await article.pagination({
        query:{sex: "male",city: "Kyleighbury"},
        page:1,                         //Current page, default 1
        limit:10,                       //Number of displays per page, default None
        sort:{createTime:-1},         
        lookup:[
          {
            from: "user",
            let: {
              userId: {
                $toObjectId: "$userId"
              }
            },
            pipeline: [
              {
                $match: {
                  $expr: {
                    $eq: [
                      "$_id",
                      "$$userId"
                    ]
                  }
                }
              },
            ],
            as: "userInfo"
          }
        ],        
    },{maxTimeMS:5000});
}

import { faker } from '@faker-js/faker';
export default async (request)=>{
  const article = request.mongo.client.db('demo').collection('article');
  const user = request.mongo.client.db('demo').collection('user');
  
  // Randomly add 100w
  const insertUserData = async ()=>{
    for(let i=0;i<100;i++){
      const arr = [];
      for(let i=0;i<10000;i++){
        arr.push({
          bigDataId:"6735dfab1b94ab6f346bb426",
          name:faker.person.fullName(),
          email:faker.internet.email(),
          avatar: faker.image.avatar(),
          birthday: faker.date.birthdate(),
          sex: faker.person.sexType(),
          phone:faker.phone.number(),
          city:faker.location.city(),
          country:faker.location.country(),
          zipCode:faker.location.zipCode(),
          streetAddress:faker.location.streetAddress(),
          timeZone:faker.location.timeZone(),
          createTime:new Date().getTime()
        });
      }
      await user.insertMany(arr);
    }
  }
  // // Add 100000 users at random
  insertUserData();
  
 
  // Insert 10 million data
  const data = await user.sample({size:10,project:{_id:1,name:1}});  // Randomly obtain 10 pieces of user data
  const insertBigData = async ()=>{
    for(let i=0;i<1000;i++){
      const arr = []
      for(let i=0;i<10000;i++){
        const userId = String(data[Math.floor(Math.random() * data.length)]._id);
        arr.push({
          userId,
          name:faker.person.fullName(),
          email:faker.internet.email(),
          avatar: faker.image.avatar(),
          birthday: faker.date.birthdate(),
          sex: faker.person.sexType(),
          phone:faker.phone.number(),
          city:faker.location.city(),
          country:faker.location.country(),
          zipCode:faker.location.zipCode(),
          streetAddress:faker.location.streetAddress(),
          timeZone:faker.location.timeZone(),
          createTime:new Date().getTime()
        });
      };
      await article.insertMany(arr);
    }
  }
  insertBigData();
  article.createIndexes([{sex: 1, city: 1,userId:1}]);  //Creating Indexes
  return article.count();
}

Next

SSE Communications