Exploration of Serverless multi environment configuration scheme

I believe that after reading the previous several articles about Serverless Component, we have experienced the traversal it brings to our development. But in our daily development projects, it is not just a simple project deployment. In our agile development process, there are keywords such as development, joint debugging, testing, pre release and formal environment. So there are doubts about small partners. After my business development, how to manage the configuration of different environments? For example, how to switch between database configuration of test environment and formal environment? So I put it into practice and wrote this article to study and discuss with you.

After reading this article, you will learn:

  1. Principle of Serverless Component deployment
  2. Basic use of dotenv module
  3. How to switch multi environment configuration based on dotenv
  4. How to refine the general configuration in serverless.yml

Serverless Component

Theory guiding practice

Before introducing the method, we need to make a brief introduction to the principle of Serverless Component deployment. What happens when we configure the project in the serverless.yml file and execute the sls --debug command?

The core steps are as follows:

1. Initialize context: including analyzing component dependency tree, injecting environment variables through dotenv, etc.
2. Install dependent component modules: different from 'npm install', serverless component will download and decompress the npm module specified by 'component' into '~ /. Serverless / components / registry / npm / @ serverless / x x x @ x.x.x'.
3. Execute the 'default' function of the component module: this default function is the deployment logic code provided by the developer, such as uploading the packed and compressed code to cos, and then deploying it to scf.

This article only needs to care about the environment variable injection in the first step.

It can be found that the Serverless Framework deployment command will inject the environment variables in the. env file into the deployment process by default, which is why when we use Tencent cloud components, we need to create an. env file with the following contents:

TENCENT_SECRET_ID=xxx
TENCENT_SECRET_KEY=xxx

Note: of course, Tencent cloud components support one click code scanning login, if you do not want to configure. env files. Is it cool~

Based on this, we can do a lot of things with. Env files. For example, in serverless.yml, the injected environment variables can be obtained by ${env.xxx}.

dotenv module

Dotenv Is an ability to inject environment variables into. env files process.env Module.

The specific use is very simple. First install npm install dotenv --save, and then import it into your project entry file:

require("dotenv").config();

Manage multi environment configuration

Having said so much, I finally got to the main point of this article. Here to tencent-koa For example, let's initialize our project:

# This command will copy the github directory containing 'serverless.yml' as the project template to the local
$ serverless create --template-url https://github.com/yugasun/tencent-serverless-demo/tree/master/serverless-env

# Installation dependency
$ cd severless-env && npm install

Then create two new profiles:

# . env.test file content
USER_NAME=yugasun_test
USER_EMAIL=yugasun_test@163.com

# . env.release file contents
USER_NAME=yugasun_release
USER_EMAIL=yugasun_release@163.com

Then create a new entry file app.js:

const dotenv = require("dotenv");
const Koa = require("koa");
const KoaRouter = require("koa-router");

const { CODE_ENV } = process.env;
dotenv.config({
  path: `${__dirname}/.env.${CODE_ENV}`
});

const app = new Koa();
const router = new KoaRouter();

router.get("/", async ctx => {
  ctx.body = {
    name: process.env.USER_NAME,
    email: process.env.USER_EMAIL
  };
});

app.use(router.allowedMethods()).use(router.routes());

// don't forget to export!
module.exports = app;

This is a simple demo, which reads different configurations of. env.xxx through the environment variable code ﹣ env injected by the cloud function, so as to realize the configuration switching of different environments. The core code is:

const { CODE_ENV } = process.env;
dotenv.config({
  path: `${__dirname}/.env.${CODE_ENV}`
});

Note: here, the config function of dotenv can specify path as the target. env file path.

So you only need to configure an environment variable CODE ENV for the cloud function. Next, we will write the serverless.yml file:

# Current operating environment
CODE_ENV: test

MyExpress:
  component: "@serverless/tencent-koa"
  inputs:
    region: ap-guangzhou
    functionName: express-function
    code: ./
    functionConf:
      timeout: 10
      memorySize: 128
      environment:
        variables:
          CODE_ENV: ${CODE_ENV}
    apigatewayConf:
      protocols:
        - http
        - https
      environment: ${CODE_ENV}

We all know that environment variables can be configured through functionConf.environment.variables. Not only the environment variable of cloud function is configured here, but also apigatewayConf.environment is configured to distinguish the testing and publishing environment of API gateway.

Tip: you can define the common variable code Ou env at the top of the yml file, and then reference the variable through ${code Ou env}.

Then execute the deployment command sls --debug. After the deployment is successful, visit the url link to create a successful url, and you can see the result of the configured environment variable:

{
  "name": "yugasun_test",
  "email": "yugasun_test@163.com"
}

When we have finished the development, we need to deploy to the release environment, just change the code [env] value in serverless.yml to release, and then redeploy.

Configuration optimization 1

Careful partners will find that: Although successfully deployed cloud functions can successfully read different environment configurations, each deployment will upload two configuration files, namely. env.test and. env.release. Sometimes we don't want to expose the configuration of the production environment in the test environment, so we need to upload only the corresponding configuration file every time we deploy. To do this, just add the exclude and include configurations in the serverless.yml configuration file:

CODE_ENV: release

MyExpress:
  component: "@serverless/tencent-koa"
  inputs:
    region: ap-guangzhou
    functionName: express-function
    code: ./
    exclude:
      - .env.release
      - .env.test
    include:
      - .env.${CODE_ENV}
    functionConf:
      timeout: 10
      memorySize: 128
      environment:
        variables:
          CODE_ENV: ${CODE_ENV}
    apigatewayConf:
      protocols:
        - http
        - https
      environment: ${CODE_ENV}

Here, all configuration files are ignored through the exclude configuration, and then the specified configuration files are included through the include configuration. The reason for this is that we specified the code field as. / project root, so all files in the project root will be uploaded by default.

Configuration optimization 2

Of course, you can also write any fixed parameters in serverless.yml to the. Env file, such as the CODE_ENV variable here, and then use ${env.CODE_ENV} reference. For example, we write the code ﹣ env to. Env:

CODE_ENV=release

Then modify the serverless.yml configuration:

MyExpress:
  component: "@serverless/tencent-koa"
  inputs:
    region: ap-guangzhou
    functionName: express-function
    code: ./
    exclude:
      - .env.release
      - .env.test
    include:
      - .env.${env.CODE_ENV}
    functionConf:
      timeout: 10
      memorySize: 128
      environment:
        variables:
          CODE_ENV: ${env.CODE_ENV}
    apigatewayConf:
      protocols:
        - http
        - https
      environment: ${env.CODE_ENV}

It should be noted that there is a disadvantage of. env files, that is, they cannot define objects and arrays. But for the private configuration, it's better to put it in. env, so you can ignore the deployment based on the file.

Configuration optimization 3

When the same business code needs to be deployed to different regions, but the function parameter configuration and API gateway configuration are consistent, how to configure it? The general configuration can be extracted as follows:

# global config
CODE_ENV: release

# function config
FUNC_CONF:
  name: express-function
  memorySize: 128
  timeout: 10
  environment:
    variables:
      CODE_ENV: ${CODE_ENV}

# apigw config
API_CONF:
  protocols:
    - http
    - https
  environment: ${CODE_ENV}

# guangzhou service
MyExpressGZ:
  component: "@serverless/tencent-koa"
  inputs:
    region: ap-guangzhou
    functionName: ${FUNC_CONF.name}
    code: ./
    exclude:
      - .env.release
      - .env.test
    include:
      - .env.${CODE_ENV}
    functionConf: ${FUNC_CONF}
    apigatewayConf: ${API_CONF}

# beijing service
MyExpressBJ:
  component: "@serverless/tencent-koa"
  inputs:
    region: ap-beijing
    functionName: ${FUNC_CONF.name}
    code: ./
    exclude:
      - .env.release
      - .env.test
    include:
      - .env.${CODE_ENV}
    functionConf: ${FUNC_CONF}
    apigatewayConf: ${API_CONF}

How to select configuration scheme

This paper introduces two schemes:

  1. Configuration via. env
  2. By defining variables in serverless.yml

They can all define global variables, so how to choose to use them in actual development?

Note: variables defined by serverless.yml or automatically injected into. env can only be obtained after sls --debug command is executed. The successful code of the actual deployment needs to be manually loaded by the dotenv module to specify the. env file. Of course, if you can also get the required variables by parsing the serverless.yml file, you can.

Usually, I will put the configuration related to the deployment into serverless.yml, and the business-related configuration into the. env file. Of course, it's just a personal suggestion here. How to configure it depends on your usage habits.

Other languages

Although this article only describes how to manage multi environment configuration in Nodejs project, other languages basically implement dotenv module, so this method is universal, such as Python dotenv module of python, which is basically the same in use:

# settings.py
from dotenv import load_dotenv
from pathlib import Path  # python3 only
env_path = Path('.') / '.env.test'
load_dotenv(dotenv_path=env_path)

summary

This article involves that all source codes are maintained in open source projects tencent-serverless-demo in serverless-env

This is the end of the whole article. Of course, this is just my experience in daily development. If you have a better practice, welcome to contribute to the Serverless open source community together.

Portal:

Welcome to: Serverless Chinese network , you can Best practices Experience more about Serverless application development!

Recommended reading: Serverless architecture: from principle, design to project implementation

Tags: npm github Python Database

Posted on Mon, 23 Mar 2020 06:04:28 -0400 by lalloo