BA Barış Akın~/blog

·16 min read·48 views

Cobra: A Scripting Language with Three Engines, No GIL and Native Compilation

A full tour of Cobra, a language written in Go: end-delimited blocks, structs and records, decorators, real parallelism without a GIL, channels, a rich standard library, an LSP and a debugger. How the tree-walking interpreter, the bytecode VM and a native compiler that beats PyPy work.

Cobra: A Scripting Language with Three Engines, No GIL and Native Compilation

Cobra is a general-purpose scripting language I wrote in Go. It closes blocks with end instead of indentation, ships as a single static binary, and can run the same program down three different paths: a tree-walking interpreter, a bytecode VM, and a native compiler that goes straight to machine code. There is no GIL; async tasks really do run on every core at once. Around it there is an LSP server, a formatter, a VS Code extension, a test framework, a Django-style web framework and a NumPy-like numeric library.

This post walks through the whole language: syntax, data types, functions, objects, errors, concurrency, the standard library, the tooling, and finally the internals and performance. I ran every example here with Cobra 0.13.0 on both engines; the output shown is the real output. The language's site is at cobralang.baltavista.com.

Why another language?

I built Cobra layer by layer: first the lexer, then a Pratt parser, then a tree-walking evaluator, then a bytecode compiler and stack VM, and last the native compiler. From the start there was one rule: every feature lands in both engines, and both are tested to behave identically. The tests and benchmarks require the two engines to produce byte-identical output; if they differ, the numbers are not even reported.

What I was aiming for:

  • Install-free scripts: copy one file and run it. No venv, no package manager.
  • Real parallelism: CPU-bound work should actually get faster when spread across cores.
  • Embeddable and readable: the whole language should stay small enough to read and understand end to end.
  • Speed when you need it: compile the same source natively and get ahead of JIT runtimes on numeric work.

Cobra in sixty seconds

let name = "Cobra"
let langs = ["go", "rill", "cobra"]
let ages = {"ada": 36, "linus": 28}

def greet(who, times)
    let out = []
    for i in range(times)
        out.push("hello ${who} #${i + 1}")
    end
    return out
end

for line in greet(name, 2)
    print(line)
end

let squares = map(def(x) -> x * x, [1, 2, 3, 4])
print(squares, squares[-1], squares[1:3])
print(filter(def(n) -> n % 2 == 0, range(10)))
let a, b = 1, 2
a, b = b, a
print(a, b, 7 / 2, float(7) / 2)
for k in ages
    print(k, ages[k])
end
print("  a,b  ".strip().split(","))

Output:

hello Cobra #1
hello Cobra #2
[1, 4, 9, 16] 16 [4, 9]
[0, 2, 4, 6, 8]
2 1 3 3.5
ada 36
linus 28
["a", "b"]

The rules are few and clear:

  • A newline ends a statement; there are no semicolons.
  • Blocks open with a keyword such as def, if, elif, else, while, for, struct or try, and close with end. Indentation is purely cosmetic; bad indentation never breaks a program.
  • # starts a comment that runs to the end of the line.
  • Logical operators are words: and, or, not. The empty value is null.
  • Expressions and call arguments inside ( [ { may span several lines.

Variables and types

let defines a new name in the current scope; a plain x = ... updates an existing one. Assigning to an undefined name is an error, so a typo never silently creates a new variable. const defines an immutable binding, but the immutability is of the binding, not the value: after const xs = [1], xs = ... is forbidden while xs.push(2) is fine.

Types: integers, floats, strings, booleans, null, lists, dicts, and first-class functions. A few details:

  • Mixing ints and floats promotes to float: 1 + 2.5 is 3.5, 1 == 1.0 is true.
  • / between two ints is integer division: 7 / 2 is 3. For a fractional result, float(7) / 2.
  • Falsy values: false, null, 0, "", [], {}.
  • and and or short-circuit and return the deciding operand: 0 or "x" is "x".
  • Multi-value assignment makes a swap one line: a, b = b, a. The right-hand side is evaluated fully before anything is bound.
  • += -= *= /= %= work on variables, index targets (xs[i] += 1) and fields (self.n += 1).

Collections and strings

Lists are reference types; let b = a does not copy, it points at the same list. Negative indices count from the end (l[-1]), + concatenates, == compares deeply. Methods: push, pop, contains, find, count, reverse, sort.

Slicing works like Python's seq[low:high:step]: xs[:2], xs[1:], xs[::-1]. Out-of-range bounds clamp. On lists it returns a new list, on strings a rune-based (UTF-8 safe) string, and on a range another lazy range.

Dicts keep insertion order. Keys can be strings, ints, booleans or null. Reading a missing key with d[k] is an error; check with has or use get(k, default). Methods: keys, values, items, has, get, del.

Strings are immutable and indexable. Methods such as upper, lower, strip, split, join, replace, startswith, find and count chain. ${...} embeds an expression in a string; any expression fits inside, even a nested interpolation. Triple-quoted strings cover multi-line text.

Sorting can take a comparator, and the fmt module shows numbers properly on screen:

import "fmt"
import "json"
let rows = json.parse("[{\"name\": \"keyboard\", \"price\": 1299}, {\"name\": \"mouse\", \"price\": 349.5}]")
rows.sort(def(a, b) -> a["price"] < b["price"])
for r in rows
    print(fmt.pad(r["name"], 8, " "), fmt.group(r["price"], 2))
end
   mouse 349.50
keyboard 1,299.00

sort(less) is a stable sort, so equal elements keep their input order and multi-key sorts compose by running one after another. fmt.group and math.round round halves to even (banker's rounding) and round through the decimal text rather than binary floating point, so sneaky errors like 0.145 * 100 don't happen here.

Functions, closures and decorators

Functions are first-class values and capture the scope they are defined in. One-expression anonymous functions use the arrow form: def(x) -> x * 2. For a multi-line body there is the block form def(x) ... end. map, filter, reduce, zip, enumerate, any and all are built in.

Decorators sit above a def, and @d means name = d(name). It is the natural way to register web routes:

let routes = {}

def route(path)
    return def(fn)
        routes[path] = fn
        return fn
    end
end

@route("/")
def home(req)
    return "home page"
end

@route("/hello")
def hello(req)
    return "hello ${req["name"]}"
end

for path in routes
    print(path, "->", routes[path]({"name": "ada"}))
end
/ -> home page
/hello -> hello ada

Decorators stack; the one nearest the def is applied first. They work on async def too.

Objects: struct, inheritance, record, contract

Classes in Cobra are defined with struct (class is accepted as an alias). init is the constructor and self is implicit in method bodies. Single inheritance uses <, and super reaches the parent's method. A struct body can also hold class constants (const), properties read like fields (get), assignment hooks (set) and static methods (static def).

A record is a struct that is immutable and compared by value: two records with equal fields are ==, and with returns a new copy with some fields replaced.

struct Shape
    def init(name)
        self.name = name
    end
    def describe()
        return "${self.name}: area ${self.area()}"
    end
end

struct Circle < Shape
    const PI = 3.14159
    def init(r)
        super.init("circle")
        self.r = r
    end
    def area()
        return Circle.PI * self.r * self.r
    end
end

struct Rect < Shape
    def init(w, h)
        super.init("rectangle")
        self.w = w
        self.h = h
    end
    def area()
        return self.w * self.h
    end
    get square()
        return self.w == self.h
    end
end

record Point
    def init(x, y)
        self.x = x
        self.y = y
    end
end

for s in [Circle(2), Rect(3, 3)]
    print(s.describe())
end
print(Rect(2, 5).square)
let p = Point(1, 2)
print(p == Point(1, 2), p.with({"x": 9}))
try
    p.x = 5
catch err
    print("error:", err)
end
circle: area 12.56636
rectangle: area 9
false
true Point{x: 9, y: 2}
error: line 49: cannot modify immutable record Point

The self.area() called inside Shape.describe is defined in the subclass; methods are dispatched dynamically on the instance's real type. A contract is a lightweight interface: it lists required method names, and implements(x, Drawable) tells you whether a value defines them all.

Errors: try, catch, finally

throw can throw any value, and it is caught as-is. Runtime errors are caught as messages carrying a line number. finally always runs, and return, break and continue pass through try blocks untouched.

def divide(a, b)
    if b == 0
        throw "division by zero"
    end
    return a / b
end
try
    print(divide(10, 2))
    print(divide(1, 0))
catch e
    print("caught:", e)
finally
    print("finally always runs")
end
@memo
def x()
end

The @memo at the end is left undefined on purpose. This is exactly where the two engines differ: the tree-walking interpreter runs the program up to that line and stops there, while the VM catches the undefined name while compiling and does not run a single line. The error message and line number are the same on both:

Cobra in the terminal: two engines, parallelism and native compilation

Undefined variables are not the only thing caught at compile time: assignment to builtins, a break outside a loop and reassigning a const are all rejected before anything runs.

Concurrency: async, await and parallelism without a GIL

Calling a function defined with async def returns a task immediately, and its body runs on its own OS thread. await task waits for the result, and await [t1, t2, ...] gathers them all, results in order. An error thrown in a task is re-raised at the await.

With no GIL, CPU-bound work really speeds up. Let's do the same computation first in sequence, then with four tasks:

import "time"
import "fmt"

def work(n)
    let s = 0
    let i = 0
    while i < n
        s = s + i % 7
        i = i + 1
    end
    return s
end

async def job(n)
    return work(n)
end

let t0 = time.clock()
let seq = [work(5000000), work(5000000), work(5000000), work(5000000)]
let t1 = time.clock()
let par = await [job(5000000), job(5000000), job(5000000), job(5000000)]
let t2 = time.clock()
print(seq == par)
print("sequential ${fmt.decimal(t1 - t0, 3)}s, parallel ${fmt.decimal(t2 - t1, 3)}s")

On a 10-core Apple M-series machine the VM printed sequential 0.584s, parallel 0.186s: about 3x, with no lock, no pool and no extra syntax in the code. pmap(fn, list) is the shortest form of the same idea: the same call as map, but the elements are spread across cores and the results keep their input order.

Parallelism leaves synchronizing shared state to the programmer. There are built-in tools for that:

  • Mutex() and RWMutex(): lock, unlock, try_lock, and rlock, runlock on the reader side.
  • Channel() and Channel(n): unbuffered or n-slot buffered channels as in Go; send, recv, close, try_send, try_recv.
  • select(ops): waits on several channels at once and continues with whichever is ready; given a default it doesn't wait.
let ch = Channel(4)
let done = Channel()

async def producer()
    for i in range(5)
        ch.send(i * i)
    end
    ch.close()
end

async def consumer()
    let total = 0
    let v = ch.recv()
    while v != null
        total += v
        v = ch.recv()
    end
    done.send(total)
end

producer()
consumer()
print("total:", done.recv())
print(pmap(def(x) -> x * 10, [1, 2, 3]))
total: 30
[10, 20, 30]

The global scope and the module cache are synchronized internally, so tasks touching different globals are always safe. A recoverable panic in a racing task (an out-of-range index on a shared list, say) becomes a catchable "data race in task" error instead of crashing the program. That is a safety net, not a substitute for a lock: as in Go, mutating a shared dict without one is a fatal error.

Modules and the standard library

import "lib/utils.cobra" runs the file once, caches it and binds a utils module object; as gives it another name. Import cycles are errors, and modules are read-only from outside.

The built-in native modules are written in Go and shared by both engines. Every failure in them is catchable:

modulecontents
fsread, write, append, read lines, directory operations
math, randommath functions, seedable random numbers
fmtnumbers as fixed-decimal, grouped and padded text
jsonparse that keeps key order, indented stringify
timeclock, monotonic timer, strftime-style format and parse
relinear-time regular expressions with RE2 syntax
httpclient (get, post, request) and server (serve), SSE streaming
proc, netsubprocesses, TCP sockets, TLS
crypto, encodingsha256, hmac, PBKDF2 password hashing, base64, hex
os, runtimearguments, environment, GC and memory statistics
testa built-in unit-test framework

An HTTP server is a few lines even without a framework:

import "http"
import "json"

def handler(req)
    if req["path"] == "/hello"
        let name = req["query"].get("name", "world")
        return {"headers": {"Content-Type": "application/json"}, "body": json.stringify({"hello": name})}
    end
    return {"status": 404, "body": "not found"}
end

http.serve(":8099", handler)

curl "localhost:8099/hello?name=ada" returns {"hello":"ada"}. The request dict arrives with the method, path, query, headers, body, client address and any files sent as multipart/form-data. Handlers run in parallel, one goroutine per request.

Tests live in the language too:

import "test"

@test.it
def addition()
    test.assert_eq(2 + 2, 4)
end

@test.it
def deep_equality()
    test.assert_eq([1, {"a": [2]}], [1, {"a": [2]}])
end

@test.it
def raises()
    test.assert_raises(def() -> 1 / 0)
end

let summary = test.run()
ok    addition
ok    deep_equality
ok    raises

3 passed, 0 failed, 3 total

Tooling

One binary, every mode:

cobra app.cobra                  # tree-walking interpreter
cobra --vm app.cobra             # bytecode VM
cobra --native app.cobra         # compile to machine code and run (cached)
cobra build app.cobra            # app.cobrac bytecode file
cobra build --exe app.cobra      # self-contained file with the VM embedded
cobra build --native app.cobra   # standalone native executable
cobra                            # REPL (multi-line, keeps state)
cobra fmt -w app.cobra           # source formatter
cobra watch app.cobra            # re-run when the file changes
cobra lsp                        # language server
cobra dap                        # debugger (Debug Adapter Protocol)
  • The REPL exists on both engines; while you type an open def or if block, it waits for the rest with a .. continuation prompt.
  • The LSP server gives live diagnostics from the real parser and compiler: syntax errors and undefined variables as you type, completions, hover docs, document symbols and go-to-definition.
  • The VS Code extension brings syntax highlighting, block-aware indentation, snippets, completions on module members and an outline view; it starts the LSP itself when cobra is on PATH.
  • The debugger: cobra dap offers editors breakpoints, stepping, a call stack and variable inspection over the Debug Adapter Protocol. --ast and --tokens print what the parser and lexer see, and runtime errors come with a stack trace.
  • Go plugins: import "name.so" loads a plugin written in Go at runtime and uses it like a module.

Release binaries are static with no dependencies, built for macOS (arm64, amd64), Linux (amd64, arm64) and Windows.

Internals: one front end, three engines

Cobra's architecture: lexer, parser, AST and three engines

The lexer turns source into tokens, and the Pratt parser turns tokens into an AST. From there the road splits in three, but the behaviour of values (the object package) and the standard library (the eval package) are shared by all of them. That is why a list method or a json.parse call cannot behave differently on the two engines.

The tree-walking interpreter evaluates AST nodes directly. There is no setup cost, which makes it ideal for the REPL and short scripts.

The bytecode VM runs the benchmark suite about 4x faster. What makes it fast, roughly in order of impact:

  • Per-VM allocation arenas for floats and struct instances. Safe because there is no GIL and the arenas belong to one VM.
  • A registerized dispatch loop: the instruction pointer, stack pointer and value stack live in locals and are synced back only around calls and error paths. Bytecode ends with OpHalt, so the loop needs no bounds check.
  • Eleven fused superinstructions that collapse the hottest patterns into one dispatch: compare-and-jump like while i < n, compute-and-store like i = i + 1, local.field reads, list[local] indexing.
  • Inline fast paths for int and float arithmetic, and interning of small integers (-1024 to 131071).
  • Call frames living in a preallocated array: zero allocations per call.
  • A custom open-addressing hash table for string-keyed dicts, with hashes cached on the string objects.
  • Line-table lookups moved entirely off the hot path; error messages work out their line number only when needed.

The native compiler runs whole-program type inference. Cobra is dynamically typed, but real programs are mostly type-stable: a variable always holds an int, a list always holds floats. Wherever the compiler can prove that, it emits unboxed ints and floats, flat structs, and native slices and maps, using the Go toolchain as the code generator. The output is verified byte-identical to the VM's, down to float formatting and rounding.

Native compilation covers the type-stable core of the language: functions, structs with init and methods, lists, dicts, strings, loops and math. A program that uses closures, async, try/catch, inheritance, records or other modules is reported as "not natively compilable", with each blocking construct named; that program runs unchanged on the VM. Producing a native file needs Go on the build machine; running it needs nothing (on my machine it is a static Mach-O file of about 2 MB).

Performance

Native compilation and the two engines compared

The top chart shows the measurements published on the language's site: the n-body, binary-trees, spectral-norm, mandelbrot and word-frequency benchmarks are written identically in Cobra and Python, and every runtime prints the same output. Native Cobra finishes in 0.28 seconds in total, ahead of PyPy's tracing JIT (0.44 seconds); CPython 3.14 needs 5.99 seconds for the same suite.

The bottom chart compares the two engines on six microbenchmarks: on recursive calls (fib) the VM is more than 10x faster, 4.1x in total.

To be honest, the VM doesn't beat CPython at everything. For this post I timed a 600×600 pixel, 200-iteration mandelbrot loop myself: the tree-walking interpreter took 16.5 seconds, the VM 2.5 seconds, CPython 3.14 1.7 seconds, and natively compiled Cobra 0.07 seconds (once the build had been done and cached). With heavy float arithmetic and complex loop conditions, the VM's boxing cost shows; the right tool for that kind of work is --native anyway. Cobra's real lane is fast, install-free scripts, embedded scripting, and a runtime small enough to understand end to end.

Ecosystem: cobranum and Venom

cobranum is a NumPy-style numeric-array library built on Gonum, or on Apple Accelerate (BLAS/LAPACK) when compiled with cgo on macOS. It offers dense 1-D and 2-D arrays, vectorized operations, broadcasting, reductions and linear algebra. The book covers it with scientific examples such as least squares, PCA, PageRank, numerical integration and differential equations:

The Lotka-Volterra predator-prey model with cobranum

Venom is a Django-inspired web framework written entirely in Cobra: routing by decorator, an ORM over OxiDB, a template engine with inheritance, middleware, sessions, static files and CSRF protection. Since Cobra prefers factory functions to class inheritance, a model comes out of a factory:

import "venom/router.cobra"
import "venom/response.cobra"
import "venom/orm.cobra"

let route = router.route
let db    = orm.connect("127.0.0.1", 4444)
let User  = orm.model(db, "users").ensure()

@route("GET", "/users/<name>")
def show_user(req)
    let user = User.get({"name": req["params"]["name"]})
    if user == null
        return response.not_found("no such user")
    end
    return response.ok(user)
end

router.serve(":8000")

The connection to OxiDB goes through the oxidb.so plugin written in Go. In the same family there is also an Entity Framework-style ORM for OxiDB's SQL engine (oxisqlorm).

The book

The whole language is covered in two books, in Turkish and English: from the basics to control flow, from collections to object orientation, from concurrency to the internals and cobranum. The English edition adds a chapter on Venom as well. Both are free as PDFs:

Trying it

Binaries, the VS Code extension, cobranum and the books are at cobralang.baltavista.com. On macOS (Apple Silicon) the shortest path is:

curl -LO https://cobralang.baltavista.com/releases/cobra-0.13.0-darwin-arm64
chmod +x cobra-0.13.0-darwin-arm64 && mv cobra-0.13.0-darwin-arm64 /usr/local/bin/cobra
cobra --version

Then open a file and write print("hello").

BA
Barış Akın

Software solutions architect with 25+ years of experience. FinTech and distributed systems; creator of OxiDB, Rill and Cobra.