Skip to content

glMatrix v4.0 - Request for feedback #453

@toji

Description

@toji

glMatrix was a project I started 12 years ago(!), because at the time there weren't too many good options for doing the type of matrix math realtime 3D apps required in JavaScript. I never thought that anyone outside of myself and a few early WebGL developers would use it, and certainly didn't anticipate it becoming as popular as it did.

Fortunately for everyone, the landscape for 3D on the web looks a lot different today than it did over a decade ago! There's a lot of great libraries that offer comprehensive tools for creating realtime 3D web apps, usually with their own very competent vector and matrix capabilities built in. Many of these offer features or syntax niceties that glMatrix hasn't been able to match due to it's history and design ethos.

The web itself has also evolved in that time. When I published the first version of glMatrix Chrome was on version 5, Firefox was at version 3, and Internet Explorer was still a thing developers cared about. Node.js and TypeScript hadn't even been released! We've made astronomical strides in terms of the capabilities of the Web platform since then. For example, none of the following existed (or at the very least were widespread) at the time glMatrix was first developed:

  • let and const
  • Arrow functions ((x) => { return x; })
  • JavaScript classes
    • which includes Getters and Setters
  • JavaScript Modules
  • Template literals
  • Spread syntax (someFunction(...args);)
  • Even Float32Array wasn't around until shortly after!
  • Oh, and WebGL itself was still in development

Over the years glMatrix has been updated in small ways to take advantage of some of these things, it hasn't strayed too far from it's original design. But these days we can do so much better, and despite the excellent competition that I strongly believe there's still a need for a solid, standalone vector and matrix math library.

I've had a bunch of ideas floating around in my head for how glMatrix could be updated to take advantage of the modern web platform, but haven't had an opportunity to work on it till recently. (Let's be honest, the library has needed some maintenence for a while now.) Now that I've had a chance to try some of it out, though, I feel pretty confident that it's a viable direction for the future of the API.

So let me walk you through my plans for a glMatrix 4.0! Feedback highly appreciated!

Backwards compatibility first and foremost

glMatrix has a lot of users, and they have a lot of carefully written algorithms using the library. It would be unrealistic to expect them to do complete re-writes of their code base just to take advantage of a nicer code pattern. So the first principle of any update is that backwards compatibility is always priority number one.

This doesn't mean that EVERYTHING is carried forward, mind you. I think it's appropriate for some functionality to be labeled as deprecated and for some lesser used, fairly awkward bit of the library to be dropped. (I'm looking at you, weird forEach experiment that never quite worked the way I wanted.)

But the majority of the library should be able to be used with minimal or no changes to existing code, and new features should cleanly layer on top of that existing code rather than requiring developers to make a wholesale switch from the "old way" to the "new way".

Lots of ease-of-use improvements

glMatrix was designed for efficiency, but that left a lot to be desired in terms of ease-of-use. There's only so much that JavaScript allows in terms of cleaning up the syntax, but with some slightly unorthodox tricks and taking advantage of modern language features we can improve things quite a bit.

Constructors

The current syntax for creating vector and matrix objects isn't ideal (I'll be using vec3 for examples, but everything here applied to each type in the library):

let v1 = vec3.create(); let v2 = vec3.fromValues(1, 2, 3);

We'd much rather use the familiar JavaScript new operator. Turns out we can without losing any backwards compatibility simply by declaring our class to extend Float32Array!

export class Vec3 extends Float32Array { constructor(...values) { switch(values.length) { case 3: super(values); break; case 2: case 1: super(values[0], value[1] ?? 0, 3); break; default: super(3); break; } } }

This allows us to use a few variants of a typical constructor.

let v1 = new Vec3(); // Creates a vector with value (0, 0, 0) let v2 = new Vec3(1, 2, 3); // Creates a vector with value (1, 2, 3) let arrayBuffer = new ArrayBuffer(32); let v3 = new Vec3(arrayBuffer); // Creates a vector mapped to offset 0 of arrayBuffer let v4 = new Vec3(arrayBuffer, 16); // Creates a vector mapped to offset 16 of arrayBuffer let v5 = new Vec3(v2); // Creates a copy of v2

It's pretty flexible, and not that complex to implement! And of course because Vec3 is still a Float32Array under the hood, you can pass it into WebGL/WebGPU (or any other API that expects Typed arrays or array-like objects) with no conversion.

gl.uniform3fv(lightPosition, v5);

Static methods

For backwards compatibility we'll keep around vec3.create() and friends, but have them return instances of the new Vec3 class instead of raw typed arrays. In order to keep everything together, they'll become static methods on the Vec3 class. Same goes for every other existing method for a given type.

export class Vec3 extends Float32Array { static create() { return new Vec3(); } static fromValues(x, y, z) { return new Vec3(x, y, z); } static add(out, a, b) { out[0] = a[0] + b[0]; out[1] = a[1] + b[1]; out[2] = a[2] + b[2]; return out; } static sub(out, a, b) { out[0] = a[0] - b[0]; out[1] = a[1] - b[1]; out[2] = a[2] - b[2]; return out; } }

Used as:

let v1 = new Vec3(1, 2, 3); let v2 = new Vec3(4, 5, 6); Vec3.add(v1, v1, v2); // v1 is now Vec3(5, 7, 9); Vec3.sub(v1, v1, [1, 1, 1]); // v1 is now Vec3(4, 6, 8);

As a minor design aside, I felt pretty strongly that as a class the type names should begin with an uppercase, but that does break backwards compat since in the original library all the "namespaces" were lowercase. This can be resolved by having the library defined a simple alias:

export const vec3 = Vec3;

Which then allows you to import whichever casing you need for your app, and even mix and match.

import { Vec3, vec3 } from './gl-matrix/vec3.js'; // This is all fine. let v1 = new Vec3(1, 2, 3); let v2 = new vec3(4, 5, 6); Vec3.add(v1, v1, v2); vec3.sub(v1, v1, [1, 1, 1]);

I would probably encourage migration to the uppercase variant over time, though.

Instance methods

Once we have a proper class backing out vectors and matrices, we can make many of the methods for those types instance methods, which operate explicitly on the this object.

export class Vec3 extends Float32Array { add(b) { this[0] += b[0]; this[1] += b[1]; this[2] += b[2]; return this; } sub(b) { this[0] -= b[0]; this[1] -= b[1]; this[2] -= b[2]; return this; } // etc... }

Turns out that this doesn't conflict with the static methods of the same name on the same class! And it makes the syntax for common operations much easier to type and read:

let v1 = new Vec3(1, 2, 3); let v2 = new Vec3(4, 5, 6); v1.add(v2).sub([1, 1, 1]); // v1 now equals Vec3(4, 6, 8);

Actually there's two ways of going about this. One is that you implicitly make every operation on a vector apply to the vector itself, as shown above. The other is that you have each operation return a new instance of the vector with the result, leaving the operands unchanged. I feel pretty strongly that the former fits the ethos of glMatrix better by not creating constantly creating new objects unless it's necessary.

If you don't want to alter the values of the original object, there's still reasonably easy options that make it more explicit what you're doing and where new memory is being allocated.

let v3 = new Vec3(v1).add(v2); // v3 is now v1 + v2, v1 and v2 are unchanged.

And, of course you can mix and match with the older function style too, which is still handy for applying the result of two different operands to a third value or simply for migrating code piecemeal over time.

v1.add(v2); Vec3.sub(v3, v1, [1, 1, 1]);

Attributes

Vectors being a real class means we can also offer a better way to access the components, because lets face it: typing v[0] instead of v.x is really annoying. Getters and setters to the rescue!

export class Vec3 extends Float32Array { get x() { return this[0]; } set x(value) { this[0] = value; } get y() { return this[1]; } set y(value) { this[1] = value; } get z() { return this[2]; } set z(value) { this[2] = value; } }

Now we can choose to reference components by either index or name:

let v = new Vec3(1, 2, 3); // Now this... let len = Math.sqrt((v.x * v.x) + (v.y * v.y) + (v.z * v.z)); // Is equivalent to this... let len2 = Math.sqrt((v[0] * v[0]) + (v[1] * v[1]) + (v[2] * v[2]));

All of method implementations internally will continue to lookup components by index both because it's a bit faster and because it allows for raw arrays to be passed in as temporary vectors and matrices, which is convenient.

Swizzles!

And hey, while we're adding accessors, why not borrow one of my favorite bits of shader syntax and add swizzle operators too!

export class Vec3 extends Float32Array { get xxx() { return new Vec3(this[0], this[0], this[0]); } get xxy() { return new Vec3(this[0], this[0], this[1]); } get xxz() { return new Vec3(this[0], this[0], this[2]); } get xyx() { return new Vec3(this[0], this[1], this[0]); } get xyy() { return new Vec3(this[0], this[1], this[1]); } // etc... get zzx() { return new Vec3(this[2], this[2], this[0]); } get zzy() { return new Vec3(this[2], this[2], this[1]); } get zzz() { return new Vec3(this[2], this[2], this[2]); } }

Swizzles can operate between vector sizes as well. In practice it looks like this:

let v1 = Vec3(1, 2, 3); let v2 = v1.zyx; // Vec3(3, 2, 1); let v3 = v2.zz; // Vec2(1, 1); let v4 = v1.xyzz; // Vec4(1, 2, 3, 3);

(These do break the "don't allocate lots of new objects rule a bit, but as a convenience I think it's worth it.)

Operator overloading

v1 = v2 + v3;

... is what I WISH I could implement here. Got your hopes up for a second, didn't I?

But no, JavaScript still stubbornly refuses to give us access to this particular tool because some people shot themselves in the foot with C++ once I guess? Check back in another decade, maybe?

TypeScript

I'm sure this will be mildly controversial, but I'm also leaning very strongly towards implementing the next version of glMatrix in TypeScript. There's a few reasons for this, not the least of which is that I've been using it much more myself lately as part of my job, and my understanding is that it's becoming far more common across the industry. It also helps spot implementation bugs, offers better autocomplete functionality in various IDEs, and I feel like the tooling that I've seen around things like documentation generation is a bit better.

As a result, having the library implemented natively in TypeScript feels like a natural step, especially considering that it doesn't prevent use in vanilla JavaScript. We'll be building a few different variants of the distributable files regardless.

Older Browser/Runtime compatibility

While I do feel very strongly about backwards compatibility of the library, that doesn't extend to supporting outdated browsers or runtimes. As a result while I'm not planning on doing anything to explicitly break it, I'm also not going to put any effort into supporting older browsers like IE 11 or fairly old versions of Node. Exactly where the cutoff line will land I'm not sure, it'll just depend on which versions support the features that we're utilizing.

ES Modules-first approach

glMatrix has used ES Modules as it's method of tying together multiple source files for a while, and I don't intend to change that. What I am curious about is how much demand there is out there for continuing to distribute other module types, such as CommonJS or AMD modules.

One thing I am fairly reluctant to continue supporting is defining all the library symbols on the global object (like window), since the library itself is already reasonably large in it's current form and the above changes will only make it larger.

Which brings me to a less exciting topic:

Caveats

All of the above features are great, and I'm sure that they'll be a net win for pretty much everybody, but they come with a few caveats that are worth being aware of.

File sizes will be larger

The addition of all the instance methods on top of the existing static methods, not to mention the large number of swizzle operations, would result is the library growing in size by a fair amount. I don't have numbers just yet, but I'd guess that the total size of glMatrix's distributable files growing by about 1/3 to 1/2 is in the right ballpark.

Obviously one aspect of building a well performing web app is keeping the download size down, and I don't want to adversely affect that just for the sake of adding some conveniences to the library.

I also, however, expect that most any developers that are truly size-conscious are already using a tree shaking, minifying build tool that will strip away any code that you're not actively accessing.

To that end, glMatrix's priority would be to avoid doing anything that would interfere with effective tree shaking rather than to try and reduce the base library size by only implementing a bare bones feature set.

length is complicated

One natural attribute that you'd want on a vector alongside the x, y, z, and w attributes is a length that gives the length of the vector itself, something that's already computed by vec{2|3|4}.length(v);

Unfortunately, length is already an attribute of Float32Array, and gives (as one would expect) the number of elements in the array.

We don't want to override length in our vector classes, since that would give nonsensical results in contexts where the object is being used as a Float32Array, which means that while we can retain the static length() function for backwards compat we'll need an alternative for the vector instances. I landed on magnitude (with an shorter alias of mag) as the replacement term, though I personally know I'm going to goof that one up at least once, and spend more time than I should wondering why the "length" of my vector is always 3. 😮‍💨

Vector/Matrix creation time will be slightly worse

This is a big one, as one of the most frequent criticisms leveled at glMatrix is that the creation of vectors and matrices is expensive compared to other similar libraries. This is because (for reasons that honestly escape me) creating TypedArray buffers or views is a slower operation in most JavaScript environments than creating a JavaScript object with the same number of elements/members.

My overall response to this has been that for many apps you'll eventually want to convert the results to a Float32Array anyway for use with one of the graphics APIs, and that avoiding creating a lot of temporary objects is a good practice for performance anyway, regardless of the relative cost of creating those objects. Both of those principles are still true, but it doesn't change the fact that this aspect of glMatrix is simply slower than some competing libraries.

The above proposals will not improve that situation, and in fact are likely to make it a bit worse. Having some extra logic in the extended classes constructor before passing through to the super() constructor will unavoidably add some overhead for the sake of providing a much nicer, more flexible syntax for developers.

If I were starting fresh I may very well take a different approach, but as I said at the top of this post backwards compat is very important to me for a library like this, so this is an area where I'm willing to accept this as a relative weakness of the library to be weighed against what I consider to be it's many strengths. Your milage may vary.

Accessors/instance methods will have overhead

As nice as it is to be able to access the object components by name, using the getter v.x is likely to always be a bit slower than accessing v[0] directly. Similarly some of the instance methods are likely to just pass through to the equivalent static method, especially in cases that would otherwise involve a significant amount of code duplication. For example:

export class Mat4 extends Float32Array { multiply(b) { return Mat4.multiply(this, this, b); } static multiply(out, a, b) { // Full matrix multiplication implementation. } }

While I'm not necessarily a fan of adding functionality that's known to be less efficient than it could be, in this case I think that the aesthetic/usability benefits are worthwhile. And it's worth considering that there are plenty of times where the clarity of the piece of code will be more valuable than ensuring it uses every clock cycle to it's maximum potential. (I mean, lets be realistic: We're talking about JavaScript here. "Perfectly optimal" was never in the cards to begin with.)

I am happy knowing that in cases where the difference in overhead has a material impact on an application's performance, the conversion from accessors to indices, or to calling the static version of functions directly can be made painlessly and in isolation.

Preview

The code snippets in this post are all pretty simplistic, but I've put a fair amount of effort into validating this approach already, and currently have a WIP version of a potential glMatrix 4.0 available to look through in the glmatrix-next branch of this repo. It is definitely not in a generally useable state at this point, but looking at the vec2.ts, vec3.ts, and mat4.ts files should give you a good idea of how things are likely to look when everything is done.

Most of my efforts so far have gone into ensuring that things can work the way that I wanted, toying with file and directly structure, ensuring that the generated docs are clear and useful, and learning far more than I anticipated about TypeScript's quirks. But now that I'm satisfied that it's possible I wanted to gather some feedback from users of the library before pushing forward with the remainder of the implementation, which will probably be largely mechanical, boring, and time consuming. I'll likely take a week off work at some point to finish it up.

Thank you!

Thank you to everyone who has made use of glMatrix over the last 12 years! It's been incredible and humbling to see all the amazing work that it's been part of. And an even bigger thank you to everyone who has contributed to or helped maintain the library, even when I personally haven't had the time to do so!

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions