A brief analysis of the starting process of Deno source code

Go straight to cli/main.rs Main method of:

pub fn main() { ... log::set_logger(&LOGGER).unwrap(); //Set log level let args: Vec<String> = env::args().collect(); let flags = flags::flags_from_vec(args); //Set v8 parameters if let Some(ref v8_flags) = flags.v8_flags { let mut v8_flags_ = v8_flags.clone(); v8_flags_.insert(0, "UNUSED_BUT_NECESSARY_ARG0".to_string()); v8_set_flags(v8_flags_); } let log_level = match flags.log_level { Some(level) => level, None => Level::Info, // Default log level }; log::set_max_level(log_level.to_level_filter()); //Generate future from command line arguments let fut = match flags.clone().subcommand { DenoSubcommand::Bundle ..., DenoSubcommand::Doc ..., DenoSubcommand::Eval ..., DenoSubcommand::Cache ... DenoSubcommand::Fmt ... DenoSubcommand::Info ..., DenoSubcommand::Install ... DenoSubcommand::Repl ..., DenoSubcommand::Run { script } => run_command(flags, script).boxed_local(), DenoSubcommand::Test ..., DenoSubcommand::Completions ... DenoSubcommand::Types ... DenoSubcommand::Upgrade ...; //Then, future is passed in as the first task to the Tokio scheduler for execution let result = tokio_util::run_basic(fut); ... }

There are two steps in the process at the beginning:

• Parse the command line parameters and create the corresponding future (task)
• Launch Tokio for future

We can't talk about the asynchronous concept of rust. The future of rust can be roughly understood as the promise of js. They all have similar states and state transitions:

However, promise relies on push, which means it is an event source and can actively change its state; while future depends on poll, which requires an excluder to poll for its state, that is to say, the excluder will execute the future poll method at an appropriate time. When the return state is Poll::Pending, it means that the excluder needs to wait for the next poll. When the return state is Poll::Pending, the excluder needs to wait for the next poll Return to Poll::Ready means that future has been completed, and then judge whether the error or successful completion of the next code logic is based on the returned value.
Of course, if the executor keeps polling, it will only waste performance. Therefore, in the future poll method, you can set a walker to inform the executor when to poll.
So the future poll method is the core, and the core logic of Deno is also completed in this method.

Then we jump straight to run_command method:

async fn run_command(flags: Flags, script: String) -> Result<(), ErrBox> { let global_state = GlobalState::new(flags.clone())?; //Step 1 let main_module = ModuleSpecifier::resolve_url_or_path(&script).unwrap(); let mut worker = create_main_worker(global_state.clone(), main_module.clone())?; //Step 2 debug!("main_module {}", main_module); worker.execute_module(&main_module).await?; worker.execute("window.dispatchEvent(new Event('load'))")?; (&mut *worker).await?; worker.execute("window.dispatchEvent(new Event('unload'))")?; if global_state.flags.lock_write { if let Some(ref lockfile) = global_state.lockfile { let g = lockfile.lock().unwrap(); g.write()?; } else { eprintln!("--lock flag must be specified when using --lock-write"); std::process::exit(11); } } Ok(()) }

I think every step here is very important, so it needs to be analyzed step by step

Step 1: create a GlobalState
Let's first look at the structure of GlobalState:

/// This structure represents state of single "deno" program. /// /// It is shared by all created workers (thus V8 isolates). pub struct GlobalStateInner { /// Flags parsed from `argv` contents. pub flags: flags::Flags, /// Permissions parsed from `flags`. pub permissions: Permissions, pub dir: deno_dir::DenoDir, pub file_fetcher: SourceFileFetcher, pub ts_compiler: TsCompiler, pub lockfile: Option<Mutex<Lockfile>>, pub compiler_starts: AtomicUsize, compile_lock: AsyncMutex<()>, }

According to the comment, GlobalStateInner represents the state of the entire deno program, and all worker s share this state.
The key attributes are flags, permissions, and file_fetcher (responsible for loading local or remote files), ts_compiler (ts compiler instance)
And it has only one way: fetch_compiled_module, which is responsible for loading the compiled module.
Therefore, the main functions and responsibilities of GlobalState can be summarized as: sharing parameters and permissions and loading and compiling module code.

Step 2: create main_worker
At present, there are three kinds of workers in deno: mainworker, CompilerWorker and WebWorker, and they all depend on Worker (another independent class) finally. Their relationship is as follows:

Worker will establish an independent Isolate to provide a new execution environment for js. MainWorker and WebWorker are encapsulated on the basis of worker to provide their own unique interface.

So go back to creating main_ In the logic of worker:

fn create_main_worker( global_state: GlobalState, main_module: ModuleSpecifier, ) -> Result<MainWorker, ErrBox> { let state = State::new(global_state, None, main_module, false)?; let mut worker = MainWorker::new( "main".to_string(), startup_data::deno_isolate_init(), state, ); { let (stdin, stdout, stderr) = get_stdio(); let mut t = worker.resource_table.borrow_mut(); t.add("stdin", Box::new(stdin)); t.add("stdout", Box::new(stdout)); t.add("stderr", Box::new(stderr)); } worker.execute("bootstrap.mainRuntime()")?; Ok(worker) }

Create main_ The first step of a worker is to create a State object. In fact, each worker in the deno corresponds to a State object, and the deno program itself also corresponds to a GlobalState. In the end, each woker will share the same GlobalState, or draw a picture intuitively:

The State structure of woker is as follows:

pub struct StateInner { pub global_state: GlobalState, pub permissions: Permissions, pub main_module: ModuleSpecifier, /// When flags contains a `.import_map_path` option, the content of the /// import map file will be resolved and set. pub import_map: Option<ImportMap>, pub metrics: Metrics, pub global_timer: GlobalTimer, pub workers: HashMap<u32, (JoinHandle<()>, WebWorkerHandle)>, pub next_worker_id: u32, pub start_time: Instant, pub seeded_rng: Option<StdRng>, pub target_lib: TargetLib, pub is_main: bool, pub is_internal: bool, pub inspector: Option<Box<DenoInspector>>, }

It can be seen that State mainly records some states of woker runtime (what permissions do you have, startup time, audit, etc.). In addition, this State also implements the trait of ModuleLoader, so State also needs to undertake the work of loading modules.

After the State object is created, the main worker will be formally created, but the StartupData:: deno must be called before creation_ Isolate_ The init method prepares the StartupData to be used by Isolate initialization. According to the parameters, you can choose snapshot initialization or direct code initialization (code location: $ / gen/cli/bundle/main.js ), the next step is MainWorker's new method:

pub fn new(name: String, startup_data: StartupData, state: State) -> Self { let state_ = state.clone(); let mut worker = Worker::new(name, startup_data, state_); { let isolate = &mut worker.isolate; ops::runtime::init(isolate, &state); ops::runtime_compiler::init(isolate, &state); ops::errors::init(isolate, &state); ops::fetch::init(isolate, &state); ops::fs::init(isolate, &state); ops::fs_events::init(isolate, &state); ops::io::init(isolate, &state); ops::plugin::init(isolate, &state); ops::net::init(isolate, &state); ops::tls::init(isolate, &state); ops::os::init(isolate, &state); ops::permissions::init(isolate, &state); ops::process::init(isolate, &state); ops::random::init(isolate, &state); ops::repl::init(isolate, &state); ops::resources::init(isolate, &state); ops::signal::init(isolate, &state); ops::timers::init(isolate, &state); ops::tty::init(isolate, &state); ops::worker_host::init(isolate, &state); } Self(worker) }

The creation of MainWorker is also relatively simple: the first step is to create an actual Worker, and the second step is to register various OPS (system calls) on CoreIsolate.

Continue to drill down into Worker creation:

pub fn new(name: String, startup_data: StartupData, state: State) -> Self { let loader = Rc::new(state.clone()); let mut isolate = deno_core::EsIsolate::new(loader, startup_data, false); state.maybe_init_inspector(&mut isolate); let global_state = state.borrow().global_state.clone(); isolate.set_js_error_create_fn(move |core_js_error| { JSError::create(core_js_error, &global_state.ts_compiler) }); let (internal_channels, external_channels) = create_channels(); Self { name, isolate, state, waker: AtomicWaker::new(), internal_channels, external_channels, } }

In fact, the creation of worker is relatively simple, because its attributes are relatively few, and the emphasis is on the isolate attribute. Worker is to rely on the ability of isolate to provide external code execution modules; and internal_channels and external_ The two properties of channel are based on the channel of t rust. The purpose is to provide the ability of woker to externally post message and internally post message.

Then we will look at ESIsolate. At this time, we have entered the core directory from the cli directory, which means that we have entered the heart of deno.
First of all, ESIsolate makes a layer of encapsulation on the basis of CoreIsolate, which is mainly responsible for the import create load instantiation execution of ES Module. Therefore, the name at the beginning actually exposes that it is related to ES Module.
The new method of ESIsolate is as follows:

pub fn new( loader: Rc<dyn ModuleLoader>, startup_data: StartupData, will_snapshot: bool, ) -> Box<Self> { //Step 1 let mut core_isolate = CoreIsolate::new(startup_data, will_snapshot); { //Step 2 let v8_isolate = core_isolate.v8_isolate.as_mut().unwrap(); v8_isolate.set_host_initialize_import_meta_object_callback( bindings::host_initialize_import_meta_object_callback, ); v8_isolate.set_host_import_module_dynamically_callback( bindings::host_import_module_dynamically_callback, ); } let es_isolate = Self { modules: Modules::new(), loader, core_isolate, dyn_import_map: HashMap::new(), preparing_dyn_imports: FuturesUnordered::new(), pending_dyn_imports: FuturesUnordered::new(), waker: AtomicWaker::new(), }; let mut boxed_es_isolate = Box::new(es_isolate); { //Step 3 let es_isolate_ptr: *mut Self = Box::into_raw(boxed_es_isolate); boxed_es_isolate = unsafe { Box::from_raw(es_isolate_ptr) }; unsafe { let v8_isolate = boxed_es_isolate.v8_isolate.as_mut().unwrap(); v8_isolate.set_data(1, es_isolate_ptr as *mut c_void); }; } boxed_es_isolate }

The code is not too much, mainly due to the overall responsibility of deno design is clear and each performs its own duties.
Step 1: create CoreIsolate
Step 2: register the callback of import module on v8 isolate
Step 3: establish the relationship between v8 isolate and rust

Finally, I came to CoreIsolate, and through layers of encapsulation, I finally saw this big Boss (sigh), and briefly analyzed the creation steps (because CoreIsolate's new method is too long to paste):
Step 1: initialize the v8 engine
Step 2: according to startup_data to determine whether to use script or snapshot to initialize v8's Isolate (if script is used, the initialized script will be executed before the first code execution or the first event loop)
Step 3: build CoreIsolate instance

This ends from mainwork to CoreIsolate.

So the last key point: MainWorker, Worker, ESIsolate and CoreIsolate all realize the Future trade (recall the Future at the beginning). When Tokio starts to schedule tasks:

The figure above can be understood as another way to look at the event cycle of deno. Later, we will go deep into dismantling what each step of the event cycle of deno has done.

Continue back to the process of creating the MainWorker:

fn create_main_worker( global_state: GlobalState, main_module: ModuleSpecifier, ) -> Result<MainWorker, ErrBox> { let state = State::new(global_state, None, main_module, false)?; let mut worker = MainWorker::new( "main".to_string(), startup_data::deno_isolate_init(), state, ); { let (stdin, stdout, stderr) = get_stdio(); let mut t = worker.resource_table.borrow_mut(); t.add("stdin", Box::new(stdin)); t.add("stdout", Box::new(stdout)); t.add("stderr", Box::new(stderr)); } worker.execute("bootstrap.mainRuntime()")?; Ok(worker) }

Now that the MainWorker has been built, add the standard input and output to the resource table of the worker, and execute bootstrap.mainRuntime(), but before executing the code, some initialization work needs to be done. Go to coreisolate:: shared directly_ Init method:

pub(crate) fn shared_init(&mut self) { if self.needs_init { self.needs_init = false; js_check(self.execute("core.js", include_str!("core.js"))); // Maybe execute the startup script. if let Some(s) = self.startup_script.take() { self.execute(&s.filename, &s.source).unwrap() } } }

You can see that deno will execute first core.js , and then try to execute the initialization script as mentioned before core.js What is it from Ni? Don't expand now, because this is related to the interaction between JS and rust. Now leave a pit for the next article to analyze. We just need to know that before executing any script, deno will initialize the interaction environment between JS and rust.

Back to execution bootstrap.mainRuntime(), now we can finally return to the familiar js world. The main work of mainRutime method is to define some global attribute methods (including those under compatible browser window objects and Deno objects).

The next step is to load the code, submit it to TS Compiler for compilation, and then execute the code. The key in this process is to pass it back to solve other codes imported by users:
worker.execute_module(&main_module).await?;

The next step is to trigger the load event. One of the differences between deno and node is that they are compatible with the browser's events, methods and properties. This obviously reduces the learning cost of our front-end, and makes the code we write easier to meet our expectations. Then why don't they be compatible with the browser when developing node? I think they were js and browsing at that time Many of the features of the device are either blank or experimental (node was born in 2009). Thanks to the crazy pit filling in recent years, this idea is now feasible:
worker.execute("window.dispatchEvent(new Event('load'))")?;

Next step:
(&mut *worker).await?;
This is the beginning of the deno event cycle. As long as the deno program does not exit, the logic here will always pause here.

Once we finish the event loop, the next step is to trigger the unload event, just like the browser page unloading (the dispatchEvent is synchronous):
worker.execute("window.dispatchEvent(new Event('unload'))")?;

The startup process of deno is analyzed here for the moment. Next time, we will analyze the event loop and how js interacts with rust.

18 June 2020, 04:14 | Views: 4108