Pro Handbook

What’s New
Getting Started

Visual Scripting Backend

Last Updated: November 1, 20242.8 min read

Overview

Visual scripting is implemented in 4 parts:
  • Parser \Code\Common\Public\Common\Scripting\Parser
  • Interpreter \Code\Common\Public\Common\Scripting\Interpreter
  • Compiler \Code\Common\Public\Common\Scripting\Compiler
  • Virtual Machine \Code\Common\Public\Common\Scripting\VirtualMachine
Our Sceneri script asset type is
  • .script.source for the text source file (we use LUA syntax for now)
  • .script.binary compiled binary format of .script.source
Compilation of script files is done through the asset compiler which produces these two files.

Compilation Options

  • SCRIPTING_USE_UNICODE
Switches between ngine::String and ngine::UnicodeString as script string type
Default: Disabled because we prefer strings as long as we don’t need to support unicode
  • SCRIPTING_DEBUG_INFO
  • Enables additional debug information in the compiled byte code. For now this is only line numbers.
  • Default: Enabled for now, since we are still in development (should be disabled for release)
  • SCRIPTING_DEBUG_TRACE_EXECUTION
  • Enables virtual machine opcode execution tracing. Giving detailed output of each executed instruction with the current stack and upvalues.
  • Default: Disabled should only be used when debugging VM execution bugs
  • SCRIPTING_DEBUG_GC_LOG
  • Enables detailed log output for the script garbage collection
  • Default: Disabled should only be used when debugging VM garbage collection bugs
  • SCRIPTING_VALUE_NAN_BOXING
  • Enables nan boxing for compiled script values. This reduces the ngine::Scripting::Value size from 16 bytes to 8 bytes.
  • Default: Disabled because runtime cost seems higher with the memory saving.
  • SCRIPTING_COMPONENT_USE_INTERPRETER
  • Switches the script component to use AST interpretation instead of executing compiled code
  • Default: Disabled because we prefer compiled code execution (optimized, smaller, faster)

Parser

The parser is a simple RD (Recursive Descent) parser which follows the LUA language syntax from Lua 5.4 Reference Manual
It takes the output from the Lexer (tokenizer) ngine::Scripting::TokenListType and (if successful) produces a complete AST with the root node UniquePtr

Interpreter

The interpreter takes the output of the parser UniquePtr and executes it by walking and interpreting the AST directly. This uses its own environment ngine::Scripting::Environment . For the execution a separate input ( SharedPtr ) of resolved variables is needed which can be generated from the Resolver ngine::Scripting::Resolver . The interpreter is exposed to the editor as a script component if interpretation is enabled and compiled in via SCRIPTING_COMPONENT_USE_INTERPRETER

Exposing of native functions

Create a native function class wrapper by implementing ngine::Scripting::ScriptFunction
This requires overwriting of GetArity() which should return the required amount of arguments and Call for the actual function implementation. Values can be marshalled (in both directions) through ngine::Scripting::Interpreter::Convert . It can be added to the environment via AddFunction(SCRIPT_STRING_LITERAL("name"), UniquePtr::Make())

Compiler

The compiler takes the output of the parser UniquePtr and generates self contained byte code UniquePtr that can be executed by the virtual machine

Virtual Machine

The virtual machine takes the output of the compiler UniquePtr and executes it. As environment table it uses the ngine::Scripting::EnvironmentObject which can be used to inject external values and functions.

Exposing of native functions

To expose a native function it needs a thin wrapper function with following signature:
bool MyFunction(ngine::Scripting::VMState& state, uint8, ngine::Scripting::Value* pArgs)
Values can be marshalled (in both directions) through ngine::Scripting::Convert . It can be added to the environment via ngine::Scripting::AddNativeFunctionToTable

Feedback

Please be sure to submit issues or feature requests through the embedded feedback form. In the event it is a major issue please contact us directly through Discord.

Table of Contents