How does the architecture team reconstruct the internal system

The front-end team inevitably needs to maintain some internal systems. Due to the unreasonable architecture design at the beginning, there are more and more "bad taste" codes with the increase of business complexity, which leads to the increase of cognitive and communication costs and even frequent problems. At this time, reconstruction naturally becomes a choice. However, the reconfiguration is neither on the spur of the moment nor achieved overnight. It needs careful analysis and orderly implementation. Taking the experimental platform as an example, this paper introduces the reconfiguration experience of the front end of Zhilian University.

The experimental platform is an A/B experimental ecology independently developed by Zhilian recruitment. Relying on the data platform and customized in combination with the company's business and technical characteristics, it provides rich experimental capabilities, scientific experimental mechanism and complete process management.

The Web side is developed and implemented using Ant Design Vue component library based on Vue implementation; The API layer is developed based on Node.js, pre processes, combines and encapsulates the original data returned by the back-end microservices, effectively reduces the coupling between the UI and the back-end interface, and realizes parallel development and interface change.

present situation

The overall design of UI typesetting and layout is not unified, the front-end interaction is complex, the function is redundant, and the "bad taste" code is increasing, which increases the difficulty of development and maintenance; The Api layer does not follow the mainstream RESTful Web API The standard is only responsible for the forwarding of back-end interfaces, and the logic is fully implemented in the Web layer, which does not effectively reduce the coupling between UI and Api layer interfaces, increasing the burden on the Web layer;

Based on the above reasons, we decided to reconstruct the experimental platform system to further improve its ease of use, cohesion and maintainability.

analysis

First, analyze the function and usage of the experimental platform page by page to facilitate a preliminary understanding of the next reconstruction work:

Overview page: it mainly displays the statistical information of the usage of the experimental platform since it was launched. In order to better display the statistical content and facilitate the maintenance of the data structure in the future, we decided that the data is no longer provided by the back-end interface, but calculated by our own Api layer;
Experiment list page: the list page is mainly used to display the key information of the experiment concerned by the user, so the display fields shall be simplified as much as possible and the primary and secondary layout of information shall be optimized; At the same time, it provides a fast jump entrance (direct statistics and direct debugging) to optimize the user experience; Add searchable experiment name and creator to optimize the search experience; For the experimental status, some statuses are no longer required (such as applying for release, agreeing to release, released and archived), and they also need to be compatible with the old experimental status. Therefore, we have made new adjustments to the experimental status:

  • Draft: New
  • Debugging: debugging status
  • Operation: operation, application for release, consent to release
  • Stop: discard, stop, published, archive
  • World overview page: the basic functions remain unchanged. Just refactor the code according to the principles

Variable page: after consideration, there is no need to release or restore variables, so you only need to display the list of variables in the "running" experiment;
Setting page: the main purpose is to display and add administrators, so it is not necessary to display all users, so it can be simplified to delete and add administrators;
Basic information page: the basic functions of this page remain unchanged, the page layout and typesetting are optimized, the user experience is unified, and the editing permission is uniformly controlled by the Api layer;
Statistical analysis page: the basic functions of this page remain unchanged. In order to facilitate maintenance, all statistical data are generated by Api layer calculation; In addition, after analysis, the real-time function of the market index page can be removed; Optimize the overall layout of the page and reconstruct the code;
Operation record page: the information of cloning experiment id needs to be added to optimize the user experience;
Control page: the basic functions of this page remain unchanged, the user experience is unified, and the editing permissions are uniformly controlled by the Api layer to optimize the page layout, typesetting and reconstruct the code;
Summary page: used to summarize the experimental results. This page is no longer needed after analysis;

principle

So far, according to the previous analysis, we have a preliminary understanding of the current situation of the experimental platform. Next, summarize some useful guidelines:

Layering, Web layer and Api layer should perform their respective functions:

  • The Web layer is only responsible for UI interaction and presentation;
  • The API layer follows Restful Web API standard, adopts Typescript development with strong type checking, and is responsible for all functional logic processing and permission control;

Layout, the overall layout shall be consistent with the design:

  • Natural layout to improve maintainability;
  • Standardize the sections and keep the design unified;
  • The upper, lower, left and right spacing of each section is consistent, and the sections are aligned;

Module, keep single responsibility and facilitate maintenance:

  • Split the modules according to their responsibilities and decouple them from each other;
  • Do not maintain the state (especially the global state) as much as possible, but through interaction with other modules;
  • Logic is decentralized and distributed to functional components;
  • Componentization to keep single component responsibility;
  • Non reusable components are placed under the directory of parent container components;

Development specifications, follow Zhilian front-end development specifications and customization principles:

  • Standardize the style and reduce the renewal cost;
  • Unified input and output specifications;
  • Prohibit all magic numbers, but through variables;
  • All inline styles are prohibited, but implemented through a more general Class;
  • Absolute positioning and floating are not used as much as possible, but implemented through a-layout components, standard document flow or Flex;

Process, using progressive reconstruction:

  • Progressive refactoring mode, refactoring in stages, each stage does not destroy the existing functions, and has the ability of separate release;

stage

Next, we divide the reconfiguration cycle into several different stages for orderly implementation.

Step 1: migrate js to ts

As we all know, JS is a dynamic language. It processes types dynamically at runtime and is very flexible. This is the charm of a dynamic language. However, a disadvantage of a flexible language is that it has no fixed data types and lacks static type checking, which leads to random assignment during multi person development. Therefore, it is difficult to eliminate more problems in the compilation stage. Therefore, For projects that require long-term iterative maintenance and the participation of many developers, it is very necessary to choose a language with strict types and find errors during compilation. TypeScript adopts strong type constraints, static inspection and tips of intelligent IDE, which can effectively reduce the speed of software corruption and improve the readability and maintainability of the code.

Therefore, this refactoring work starts with the migration from js to ts, which lays the language foundation for the subsequent model combing.

ts is limited to the node layer of API engineering. Because Vue2 is not friendly to ts support at the front end, the original js is still used.

Step 2: sort out the data model

This step is relatively simple. It is mainly to sort out the input and output information requested by the existing API interface, so as to lay a good foundation for subsequent sorting of data entities.
First, sort out all the pages of the experimental platform system, as shown below:

  • set up
  • variable
  • world
  • overview
  • Experiment list
  • Create experiment
  • View basic information
  • Edit basic information
  • View operation record
  • View statistics
  • control

Then, make further statistics on the API interfaces involved in each page, for example, setting page: obtain user list, add and delete users, set user roles and other API interfaces.
Secondly, summarize and sort out the requested input and output information of each API interface according to the sorting results in the previous step. For example, open the basic experimental information page, find the browser's development tool and switch to NetWork, right-click the request interface to find the Copy as fetch copy request result, as shown in the following figure:

The following shows the code structure of API interface request input and output information:

[[example]
// [grouping]: obtain the experimental grouping information list
fetch(
  "https://example.com/api/exp/groups?trialId=538",
  {
    credentials: "include",
    headers: {
      accept: "application/json, text/plain, */*",
      "accept-language": "zh-CN,zh;q=0.9,en;q=0.8",
      "sec-fetch-mode": "cors",
      "sec-fetch-site": "same-origin"
    },
    referrer: "https://example.com/exps/538",
    referrerPolicy: "no-referrer-when-downgrade",
    body: null,
    method: "GET",
    mode: "cors"
  }
);

const response = {
  code: 200,
  data: {
    groups: [
      {
        desp: "c_app_default_baselinev",
        flow: 20,
        groupId: 1368,
        imageUrl: "",
        type: "A,control group",
        vars: ["c_app_default_baselinev"]
      },
      {
        desp: "c_App_flowControl_baseline",
        flow: 20,
        groupId: 1369,
        imageUrl: "",
        type: "B",
        vars: ["c_app_flowControl_baseline"]
      }
    ],
    varName: ["gray_router_prapi_deliver"]
  },
  time: "2019-12-20 17:25:37",
  message: "success",
  taskId: "5f4419ea73d8437e9b851a0915232ff4"
};

Similarly, we sort out the input and output of the corresponding API interface requests of all pages according to the above process, and finally get the following file list:

Next, analyze the return value of each interface and extract the fields used in UI layer interaction, so as to define the basic data model. The data model should be able to intuitively display the basic composition structure of data, as shown below:

[[example]

// grouping
const group = {
  id: 123,
  name: 'A',  // group
  description: 'CAPP variable',  // describe
  variableValue: 'c_app_baselinev',   // Variable value
  preview: 'http://abc.jpg ', / / Preview
  bandwidth: 10  // flow
}

Step 3: sort out data entities

We combed the data model. Next, we need to further comb the data entities corresponding to the model according to the data model, as shown below:

[[example]

class ExperimentGroup {
  id: number
  name: string
  description: string
  variableValue: string
  previewImage: string
  bandwidth: number

  constructor () {
    this.id = null
    this.name = null
    this.description = null
    this.variableValue = null
    this.previewImage = null
    this.bandwidth = 0
  }
}

Define the calculated fields in Object.defineProperties as follows:

[[example]

class ExperimentCompositeMetric extends ExperimentMetric {
   unit: string

   constructor () {
      super()
      this.unit = ''

      Object.defineProperties(this,  {
          displayName: {
              get: () => (this.unit ? `${this.title}(${this.unit})` : this.title),
              enumerable: true
          }
      })
   }
}

At the same time, according to the output of the interface, the data entity of the unified Api interface output specification is defined as follows:

[[example]

class Result {
  error: boolean|string|Error
  data: any
  requestId: string|null
  constructor () {
    this.error = false
    this.data = null
    this.requestId = null
  }
}

Step 4: re sort out the interaction type interface

Next, according to the interaction function of the UI layer, define the input specification of the interface (including methods, paths, parameters, etc.), enrich the entities associated with the interface, and finally continuously adjust and optimize the entities according to needs in the actual development, as shown below:

[[example]

// Get indicator chart data
get('/api/v2/experiment/stats/trending', {
  params: {
    id: 123,
    type: "key",
    period: "day", // or hour
    from: "2019-12-17",
    to: "2019-12-18"
  }
})

Step 5: re carding UI

This step is mainly to re sort out the UI (page, layout, components, etc.), and define the basic display form and input and output specifications of the page:

[[example]

<template>
  <editable-section @click="onEdit">
    <h2 slot="header">grouping</h2>
    <table>
      <thead>
        <tr>
          <th>group</th>
          <th>Group name</th>
          <th>variable {{ experiment.variable.name }} Value of</th>
          <th>Preview</th>
        </tr>
      </thead>
      <tbody>
        <tr v-for="group in experiment.groups">
          <td rowspan="group.name | toRowspan">group.name | toType</td>
          <th>group.name</th>
          <td>
            <p>
              {{ group.variableValue }}
              <small>{{ group.description }}</small>
            </p>
          </td>
          <td>
            <img src="group.previewImage"/>
          </td>
        </tr>
      </tbody>
    </table>
  </editable-section>
</template>

<script>
import BaseSection from 'shared/components/EditableSection'

export default {
  props: {
    experiment: Object
  },
  filters: {
    toType (value) {
      return value === 'A' ? 'control group' : 'experience group'
    },
    toRowspan (value) {
      return value === 'A' ? 1 : this.experiment.groups.length - 1
    }
  },
  methods: {
    onEdit () {
      alert('Not implemented yet, please use V1. ')
    }
  }
}
</script>

Step 6: define the file layout

Before refactoring, we defined the basic file directory of the Web project. At the same time, according to previous experience, we also extracted common variables and methods, as shown in the figure below:

shared file storage is used to store public file resources. components file stores public component resources, api.js file stores all API URL resources, styles file stores public css file resources, images file stores public pictures, fonts file stores public fonts, etc.

In the variables.postcss file, some common css variables are defined as follows:

:root {
  --font-family--code: cascadia, pingfang sc, microsoft yahei ui light, Microsoft YaHei , arial, sans-serif;
  --font-size--super: 70px;
  --font-size--xl: 24px; /* Super large font size */
  --font-size--lg: 18px; /* Large size */
  --font-size: 14px; /* General font size */
  --font-size--sm: 12px; /* Small size */
  --space: 16px; /* Regular spacing, applicable to padding and margin */
  --space--sm: 12px; /* Small spacing */
  --space--xs: 8px; /* Ultra small spacing */
  --color--white: #fff;
  --color--black: #000;
  --color--subtle: rgba(0, 0, 0, 0.45); /* Non significant color */
  --color--message: #93a1a1;
  --color--info: #859900;
  --color--warning: #b58900;
  --color--trace: #657b83;
  --color--error: #dc322f;
  --color--normal: #268bd2;
  --color--lightgrey: #f0f2f5;
  --color--active: #1890ff;
}

In the global.postcss file, the style override and global style of the third-party library are defined, as shown below

@import './variables.postcss';
 
@font-face {
  font-family: 'Cascadia';
  src: url('../fonts/cascadia.ttf');
}
 
// Global style
html,
body {
  min-width: 1200px;
}
 
 
.text--description {
  color: var(--color--subtle);
}
 
.text--mono {
  font-family: var(--font-family)
}
 
// Third party style override
.ant-modal-body {
  max-height: calc(100vh - 240px);
  overflow: auto;
}
 
.ant-table-body td {
  vertical-align: top;
}
 
.ant-table-thead > tr > th {
  background: #fafafa !important;
}
 
.ant-table-small > .ant-table-content > .ant-table-body {
  margin: 0;
}
 
.ant-table {
  & .ant-empty {
    & .ant-empty-description {
      display: none;
    }
 
    &::after {
      content: 'There is nothing at present';
      display: block;
    }
  }
}

The template.js file is used to store the method of obtaining html templates, as shown below:

import favicon from 'shared/images/favicon.png'
 
function generate ({
  ctx, title, ...pageContexts
}) {
  const prepareDataString = Object.entries(pageContexts)
    .map(([key, value]) => `var ${key} = ${typeof value === 'object' ? JSON.stringify(value) : value}\n`)
    .join('')
 
  const template = `<!DOCTYPE html>
  <html>
    <head>
      //Custom title
      <title>${title ? `${title} | ` : ''}Zhilian experimental platform</title>
      <link rel='shortcut icon' href='${favicon}' />
      // Resource file occupancy
      ${ctx.template.placeholders.head.style}
      ${ctx.template.placeholders.head.link}
      ${ctx.template.placeholders.head.script}
      <script>
        //Incoming custom global page data
        ${prepareDataString}
      </script>
    </head>
    <body>
      // Resource file occupancy
      ${ctx.template.placeholders.body.root}
      ${ctx.template.placeholders.body.script}
    </body>
  </html>`
 
  return template
}
 
export default {
  generate
}

Similarly, we also define the file directory of Api project, as shown in the following figure:

shared is used to store public file resources. utils stores some public methods. The files in the models file are the data models we sorted out earlier.

Step 7: progressive development

Next, we can formally enter the next stage for incremental refactoring.
Priority is defined for all pages according to the difficulty of the project and the dependence and relevance of functions, as shown below:

  1. frame
  2. set up
  3. variable
  4. world
  5. overview
  6. Experiment list
  7. View operation record
  8. View statistics
  9. View basic information
  10. control
  11. Create experiment
  12. Edit basic information

Refactoring is carried out from low to high. Each stage does not destroy the existing functions and has the ability to publish separately. The current version is defined as v1 and the reconstructed version v2. It is distinguished on the url, such as v2/exps. The node layer version v1 is js, and then it is gradually replaced by the version V2 written in ts (because ts is downward compatible with js, it can exist in the project at the same time);

With the orderly progress of refactoring, we will find that refactoring becomes more and more handy. We still need to control the scope of refactoring and start the next function after completing one function test.

Step 8: pull out the common components

It is not difficult to find that this step is actually carried out at the same time as the previous step. In the reconstruction process, in order to ensure the simplicity, unity and maintainability of the code, we constantly separate the components according to the use scenarios and functions to ensure the unity of the code and interface. Which scenarios can separate the components?

  • Repeated codes used more than 3 times;
  • The usage scenarios are similar;
  • The logic is close, and the code is always updated together;

The following principles shall be followed:

  • Keep single component responsibility, high cohesion and low coupling;
  • Keep the configuration of parameters simple and flexible;
  • Keep the granularity appropriate and the number of code lines moderate;

On this basis, we have separated the common components such as layout, editing, avatar, menu component and experimental status, which greatly reduces the tedious and repetitive workload, as follows:
BaseLayout: the main frame layout container. The left navigation bar, the Header on the right and the section container on the right can be nested.
Header: Top layout, with default style, Title on the left, operation buttons on the right, etc.
BaseSection: a read-only layout container with a title and a nested content area at the top left.
EditableSection: an editable layout container, which can set a title on the upper left, an operation button on the upper right, and a nested content area under it.

Step 9: unify the overall layout, interactive experience and prompt information

With the progress of refactoring, we need to further optimize some details of the UI to form a unified design system:

  1. Check / adjust the overall layout, navigation and the proportion of each section, for example:

    • spacing
      The upper, lower, left and right spacing of each section is consistent, and the sections are aligned;
      The internal padding of the section is uniform;
      Uniform spacing of the same elements;
    • layout
      The position of the same component (e.g. left or right) is unified;
      The left navigation menu and the right content mode layout are adopted, and the BaseLayout component is used;
      The Header component is used at the top of the right side of the page;
      The read-only content module uses the BaseSection component, and the EditableSection component with editing;
    • form
      All tabular data shall be in compact mode;
      Lock the first row of the table and the first column of some tables;
      For tables that return all data at one time, pagination is not used in principle;
      For tables that must have pagination, the pagination area shall be displayed in the visible area;
      The main field of the table content automatically sets the column width, and the column width of the secondary content sets the minimum width;
  2. Interactive experience, such as;

    • The EditableSection component is used for form editing. Click the "Edit" button in the upper right corner to pop up the modal box for editing;
    • "Operation button" is on the right side of Header;
    • Pop up prompt box for deleting PopConfirm component;
    • The toast prompt box uses the Tooltip component;
    • Unified message prompt, title, click Details to display the specific information of the error;
  3. Standard font: the font size, color and font style of the same level and scene are unified, for example:

    • Font size of the header at the top: 24px;
    • Title of section font size: 20px;
    • Set wide font text--mono for statistics, English variables, etc;
    • Set the style text – description for auxiliary text content such as description;
  4. Unified message prompt

    • Mode of use;
    • Interactive form;
    • Display information;

Step 10: go online v2 page gradually

In order to prevent the newly launched v2 page from being used normally due to errors in some cases, we have added the "version switching" function in the upper right corner of the page, which can quickly switch to the v1 page without affecting the normal use of users. In addition, the v2 function is gradually optimized after a period of self-test and user feedback.

Step 11: launch all v2 versions

After all v2 pages and APIs are online, through a week's self-test and user feedback, there are no functional problems, and the version "switch button" in the upper right corner can be removed.

Step 12: delete v1 version

After one month of operation observation, the v2 version runs stably online and has no obvious functional problems. After that, the v1 page and code are offline. Since then, most of the reconstruction work has been completed.

Step 13: pull out the common component library

Now the reconstruction work is coming to an end, and the experimental platform has been completely changed from the previous architecture design, coding specification, layout and user experience.

Before announcing the completion of the reconfiguration, in order to further unify the user experience and improve the system iteration efficiency, we extracted a public component library. Public components are business independent and can be used in other scenarios.

Thus, the prototype of AntNest component library came into being. It is implemented based on Ant Design Vue component library, which covers public components with functions such as layout, container, card, prompt, editing and data display. v1.0.0 version will be released soon. Firstly, it will be applied and replaced on the experimental platform, Ada workbench and magic cube management system, and will be continuously optimized and operated after observation for a period of time It has been gradually popularized and used in other internal systems of Zhilian. At present, it has been successfully applied to more than a dozen projects, such as Kunpeng, Fuxi, operation platform, performance monitoring platform, etc. in the future, other internal systems will be replaced with AntNest components to form a unified management system system and realize a truly unified user experience.

Step 14: provide project template

In order to further simplify the development process and facilitate you to quickly create projects, we also provide lightweight templates based on Web Engineering and Api engineering.

Among them, the Web engineering template is based on AntNest component library. In addition to the basic Web framework structure, we also have built-in overview pages (general overview and waterfall flow overview), list pages, details pages (read-only and editable), layout pages and error pages commonly used in the management system to meet the basic needs of users.

The Api project template is based on Node.js and is developed by Typescript with strong type checking. The file directory structure consistent with the experimental platform is prefabricated to facilitate rapid development.

summary

Looking back on the whole reconstruction process, we will find that the first thing we do is not coding, but an in-depth analysis of the current situation. In this process, seeking common ground while reserving differences, some patterns will naturally appear, which are the "material" of reconstruction.

In real coding, we adopt a gradual strategy to decompose the whole process into multiple steps. We strive to ensure that after each step is completed, the whole module can meet the release standard. This means that the changes involved in each step need to be limited to a controllable range, and each step needs to include a complete test.

The above is the process and experience of the reconstruction of the experimental platform, hoping to provide help and reference for the development of new projects or the reconstruction of old projects in the future.

Posted on Fri, 03 Dec 2021 05:02:17 -0500 by aquilla