Spinnaker section 7 - Orca code details

This Spring Festival holiday has been held at home because of the new crown. I have nothing to do to turn out the spinnaker code and review each microservice again. I made some notes and shared them today. If there is spinnaker development or bug repair in the future, I can directly focus on it. It is planned to introduce the core micro services in several phases: orca, clouddriver and deck. Today, Orca will be introduced first.

Orca term explanation


     Orchestration
Parallel to the pipeline, I personally understand that orca is a microservice that supports the pipeline workflow and extends the ORCHESTRATION to connect temporary task packages
     Execution
One execution. There are two types: PIPELINE and ORCHESTRATION. The core attribute number is a set of stage s
     Stage
orca is the most complex type, associated with one execution and multiple task s. It also maintains important information such as parentStage, downStreamStage, output, etc
     StageContext
Drive the environment for this phase to provide the input required for the component steps
     Task
The component of Stage, the smallest operation unit in orca
ExecutionRunner executor of Execution
The executionrepository executor, our redis, also carries the retrieval query
Executionlauncher the initiator of Execution, receives json and encapsulates it as Execution for storage and startup
1. The Execution of JSON package can also be divided into PIPELINE and architecture, which are essentially the same
2. Final start implementation:

	    public Execution start(Execution execution) throws Exception {
			executionRunner.start(execution);
			return execution;
		}

Orca web subproject (portal)


  controller
OperationsController: the controller related to Execution
      /orchestrate*
Finally, start PIPELINE, and start execution in the form of PIPELINE
      /plan*
It will not actually execute. After a series of verifications, it stores the executor through the ExecutionRepository
      /ops*
Finally, start task, and start executor in the form of ORCHESTRATION
    TaskController:  
      /pipelines*
For pipeline stop, repeat and query, the final mapped object type is execution
      /task*
For task stop, repeat and query, the final mapped object type is execution
Orca web summary: both operations controller and task controller seem to have overlaps, but the focus is different. Operations controller focuses on creation and startup, while TaskController focuses on subsequent pauses, cancellations and queries.  

Orca core subproject (abstraction layer)


Root directory:
Define a series of tasks: retryableTask, cancelableTask, etc
annotations custom sync, used for annotation synchronization, can be used as learning reference of custom annotation
clouddriver.utils obtains information of various clouddrivers according to stage
commands don't mean much. You don't need to see them
config: orca configuration
Exceptions: in addition to defining exceptions, the Handler for handling exceptions is also defined

/*
 * Copyright 2014 Netflix, Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package com.netflix.spinnaker.orca.exceptions;

import java.util.Collections;
import java.util.Map;
import com.google.common.base.Throwables;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.core.annotation.Order;
import static java.lang.String.format;
import static org.springframework.core.Ordered.LOWEST_PRECEDENCE;

@Order(LOWEST_PRECEDENCE)
public class DefaultExceptionHandler implements ExceptionHandler {
  private final Logger log = LoggerFactory.getLogger(getClass());

  @Override public boolean handles(Exception e) {
    return true;
  }

  @Override
  public ExceptionHandler.Response handle(String taskName, Exception e) {
    Map<String, Object> exceptionDetails = ExceptionHandler.responseDetails("Unexpected Task Failure", Collections.singletonList(e.getMessage()));
    exceptionDetails.put("stackTrace", Throwables.getStackTraceAsString(e));
    log.warn(format("Error occurred during task %s", taskName), e);
    return new ExceptionHandler.Response(e.getClass().getSimpleName(), taskName, exceptionDetails, false);
}
}


  events:
Various events of execution, stage and task are defined,
The basic listening class for the above events has also been set. In essence, it doesn't do the processing by itself. It's all left to delegate for processing. Agent mode

  private void onTaskStarted(TaskStarted event) {
    Execution execution = retrieve(event);
    List<Stage> stages = execution.getStages();
    stages
      .stream()
      .filter(it -> it.getId().equals(event.getStageId()))
      .findFirst()
      .ifPresent(stage ->
        delegate.beforeTask(
          persister,
          stage,
          stage.getTasks().stream().filter(it -> it.getId().equals(event.getTaskId())).findFirst().get()
        )
      );
  }


  listeners:
Two interfaces of ExecutionListener and StageListener are defined, that is, delegate in events. Some phases and methods to be rewritten are defined. Such implementation mechanism is like echo notification.

/*
 * Copyright 2016 Netflix, Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License")
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package com.netflix.spinnaker.orca.listeners;

import com.netflix.spinnaker.orca.ExecutionStatus;
import com.netflix.spinnaker.orca.pipeline.model.Stage;
import com.netflix.spinnaker.orca.pipeline.model.Task;

public interface StageListener {
  default void beforeTask(Persister persister,
                                                Stage stage,
                                                Task task) {
    // do nothing
  }

  default void beforeStage(Persister persister,
                                                 Stage stage) {
    // do nothing
  }

  default void afterTask(Persister persister,
                                               Stage stage,
                                               Task task,
                                               ExecutionStatus executionStatus,
                                               boolean wasSuccessful) {
    // do nothing
  }

  default void afterStage(Persister persister,
                                                Stage stage) {
    // do nothing
  }
}


  locks
It is designed for multi node deployment. At this stage, I don't care
  pipeline
The core class, StageDefinitionBuilder, is inherited by all stages
    AcquireLockStage,CheckPreconditionsStage,EvaluateVariablesStage,ReleaseLockStage,WaitStage 

Orca clouddriver subproject (core implementation)


First, understand three services: oort, mort and kato, all of which point to the restful of clouddriver
oort is the big head: docking cluster, image, instance, loadbalance and other cloud platform resources
mort is a small head: docking with vpc, security group and account authentication
kkato is auxiliary: docking other auxiliary resources
    pipeline:
For task assembly in stage, note that the provider package is reserved for the vendor's own privatization, so the differentiation of orca cloud vendors is also concentrated here.
    tasks:
The specific task implementation is also the focus of the future. Each vendor in the provider has its own privatization

 

 

 

 

 

169 original articles published, 188 praised, 410000 visitors+
His message board follow

Tags: Apache Java JSON Spring

Posted on Mon, 10 Feb 2020 00:47:24 -0500 by whiterecluse