Thursday 4 October 2018

Performance Improvements for Omber

Omber's performance has always been slower than other vector drawing tools because

  • it can run in a web browser, and web browser code is slower than other code
  • it uses a custom graphics engine that I've written from scratch. Although this engine has support for Omber's gradients, it lacks many of the optimizations that are in older, more developed vector graphics engines
  • I've focused on adding features instead of on performance

This hasn't really been a problem for my uses because Omber defaults to a low quality mode that is fast enough to work with, and I can turn on the high quality HQ mode when I want a final image. I occasionally hire artists to give feedback about Omber, and I've been worried that they are initially confused by the low quality of Omber's default rendering. When they do find the HQ mode, they then tend to leave it on or they tend to regularly switch it on and off, meaning they experience sluggish performance when working.

This has worried me for a while, so I've decided to focus on improving Omber's performance enough that it can default to being in HQ mode. Below is a simulated animation of the different performance enhancements that I've made:

On the right is the original Omber. When you move a shape, it takes a while to draw, so it feels jerky and unresponsive. I spent a lot of time fixing up the performance of the initial drawing of shapes, so that Omber is responsive during shape manipulation. The middle animation demonstrates the effect of these performance improvements.

Omber also has a separate gradient generation stage that works in the background to generate gradients after you manipulate shapes. Since this runs in the background, it doesn't affect the responsiveness of Omber, so it usually isn't performance critical. Unfortunately, in HQ mode, the shapes that Omber has to deal with are more complicated, and the gradient generation started to become too slow for complex drawings. In the middle animation, you can see that after the shape has finished moving, it takes a while for the gradients to be calculated.

So I'm now working on improving gradient generation speed, so that Omber will behave like in the left animation, where it's responsive when you move a shape, and then the gradient quickly fills in afterwards.

The key to all these performance improvements is a new feature added to web browsers called Web Assembly. The existing Omber code has already been heavily optimized in the past. Unfortunately, in the past, web browsers could only run code written in JavaScript, and there are limitations to how fast those web browsers can run that code. WebAssembly lets browsers run code written in C++. This opens up many new possibilities for performance enhancements. C++ gives programmers more control over memory layouts, data types, and memory allocation. This makes it possible to rewrite key pieces of Omber code to make use of this extra control. For example, many of Omber's algorithms need to process thousands of points and triangles. With C++, these points and triangles can be stored compactly in arrays, which can be accessed more quickly and take less memory. Putting things in arrays also provides an arbitrary and implicit ordering on objects, providing a means to sort these objects. Supplementary information about these objects can be stored in arrays instead of dictionaries. Traditionally, the main problems with C++ is that it's hard to write and it's prone to security problems. I'm dealing the coding difficulties by focusing my efforts on only the small sections of code in Omber that take the most time. WebAssembly eliminates most security problems with C++ by adding extra security checks throughout the code.

As I've rewritten the slow parts of Omber in C++, I've also discovered a few coding mistakes that have reduced performance that I've eliminated. For example, I added some special code for handling some edge cases in the Edge web browser. Unfortunately, this resulted in mouse events being processed twice, halving performance.

These performance improvements will be released in the next week or two. After this round of optimizations, I think there are only limited means available for further speed-ups, such as taking advantage of increased parallelism (not really possible in web browsers) and algorithmic improvements (I have a few ideas in mind here, but those changes would require extensive new research).

Thursday 23 August 2018

Game Engine Import for Omber

One of my long-term goals for Omber is to develop vector graphics tooling for animation and games. That's why the Omber gradient engine is based on a 3d graphics pipeline typical for games instead of a more typical vector graphics pipeline. Omber has supported COLLADA and glTF export for a while because I've always wanted Omber's art to be usable in games.

Unfortunately, support for COLLADA and glTF is pretty rudimentary in most game engines. The situation is not improving either. The new glTF 3d format is one of the best 3d file standards that's been created in decades. It is the first 3d file format standard with the technical capabilities needed for sharing modern 3d models between programs. But most 3d tools have been very reluctant to add support for it. So even though Omber has been able to export its vector art for use in games, no game engines have been able to actually import this vector art.

So I took some time out from working on Omber to create some importer plugins for the game engines PIXI.js (which is also used by RPG Maker MV) and Unity. You don't have to wait around for game engines to catch up to importing modern 3d file formats. You can import Omber vector art immediately into these game engines using the importer plugins with a minimum of hassle. I even game jammed with Omber's PIXI plugin to make sure everything worked, and it was fine. The vector art exported by Omber tends to be heavy in triangles currently, which can be graphics intensive if you have hundreds of pieces of vector art on-screen at the same time, but it's fine for more normal situations. I also used the Unity plugin to make an animation from one of Omber's clipart pieces. Basically, I broke down the chibi boy image into parts and exported each one separately. After importing them into Unity, I reassembled them hierarchically so that the the legs, arms, and heads would be children of the torso object, etc. Then, I could use Unity's animation tools to make a simple run animation. If you want support for other game engines, just send me a message and I'll see what I can do.

I'm now back to working directly on Omber again. I'm not sure which areas I should focus on next. I could add in more general vector design functionality. Although Omber has some unique features, it will take a while for it to be competitive in the general graphic design space. Even then, it would be hard to compete with better resourced firms. I was thinking of making an iPad port, but now that Affinity Designer is out and Vectornator has been made free, I'm not sure there's much of a market there. Ports are also time-consuming and other versions of Omber won't see any improvements when I'm doing the port. Oddly enough, three years ago, I made a rudimentary animation tool in a few months called Animation Companion, and even now, that tool often has more traffic and more users than Omber. I could use some of Omber's technology to make a much better animation tool, something comparable to Spine and that could also handle vector art exported by Omber. I wonder if that would be a better area to focus on.

Wednesday 11 April 2018

Initial Text Support in Omber

I obviously use Omber for all my vector drawing needs. There's been a couple of times where I've had to make some graphics with some text in it though, and I had to use another tool because Omber didn't support text. I've also noticed that a lot of designers use vector drawing tools for making UI mock-ups, which also requires text support. So I decided to add some basic text support to Omber.

I was a bit torn about implementing it. A long time ago, when I was an intern at Sonic Foundry, one piece of advice they often gave was that if you can't implement a feature in an outstanding way, then don't bother implementing it. Reviewers and other users will see that one mediocre piece of the whole application, and instead of seeing it as a useful, free addition to the program for cost-conscious users who can't afford to buy another program just for that functionality, they'll instead seize on it as a weakness of the application that makes it inferior to others. This is especially true with Omber. Omber is a vector drawing tool for making drawings with advanced gradients and subtle shading. There are other vector drawing tools that are better for general purpose vector drawing tasks. If you need to make art with crazy gradients, then Omber is the better tool by far. I would argue that it even rivals tools like Illustrator. By adding text support, I would be making Omber look more like a conventional general purpose vector drawing tool, and users might get confused and think that it's worse than other programs when it's actually just used for different purposes. Would it be better for me to double-down, forget the text, and spend my time making the gradient support even stronger? In the end, I became annoyed that there are apps that are more popular than Omber but which are far worse than Omber at general purpose vector drawing tasks except that they support text, so I decided to spend some time implementing text to see what would happen.

It took me about a month and a half to implement very basic text support in Omber. Text support is a little strange in that it can be very quick and easy to implement or very long and hard to implement depending on what features you eventually want your text support to have. For example, if you only want to support English text in a text box and if you don't care if text is rendered slightly differently on different platforms, then you can just use the existing text input widgets and text drawing code to quickly implement support for that. Proper support for international text might involve huge lookup tables that would be too hard to download for a web app and custom text editing code while proper support for text transformations and kerning likely requires a custom font rendering engine.

In the end, I wanted to have the most flexibility in the underlying engine as possible, so I opted to do all the font rendering and display in Omber. This means that text in Omber will initially have fewer features and be rendered with lower quality than in other programs, but it will have consistent font rendering across platforms and the power to eventually have more interesting text effects. Right now, font rendering at small sizes isn't great, and very, very thin fonts don't quite render correctly, but with a custom font engine, I have a lot of knobs I can turn to improve things over time.

One surprisingly time-consuming aspect of doing your own font rendering is actually in font enumeration: finding out what fonts are on a system. Different platforms have different ways of listing their fonts and getting at font data, and often the documentation on them often isn't great or they use some ancient legacy approach to categorizing fonts that's really complicated. In many cases, the font information provided by the operating system isn't very good, so you have to open the font file yourself and read out extra information about the font directly. Then there's Android which doesn't even provide a way to find out what fonts are on a phone, so you just have to search around the filesystem for font files.

So in the end, I do have basic text support working in Omber. You can make some drawings with some simple text in them! Right now, I'm trying to figure out what to work on next: adding more features to the text support, better support for non-Western languages, improving the workflow for gradients some more, printing support, or something else entirely.

Wednesday 7 February 2018

Wrinkle-Free Engine Preview for Omber

I've made some good progress on the new Wrinkle-Free gradient engine for Omber. There's still a lot more work to do, and it will take several months to stabilize it and to get rid of all the bugs. The results are so good, though, that I've decided to release a preview of the Wrinkle-Free engine in the latest version of Omber. The new engine can be enabled from the Options menu.

I'm a little embarrassed to say that I've always been a little unhappy with the images created in Omber. Although Omber was still a great improvement over other vector drawing tools, I've always been reluctant to evangelize it really hard. Every time I looked at images rendered in the old engine, all I could see were the minor visual artifacts at the seams between shapes. But no more! With the new Wrinkle-Free engine, those artifacts are gone, and I can whole-heartedly say that Omber is amazing and that everyone should use it.

Below is one of the old clipart images in Omber. By default, the shape segments in the drawing are not welded together. The Wrinkle-Free engine only comes into play for welded segments, so, by default, the drawing will be rendered with the old engine. The dog on the left is rendered with the old engine. There are subtle visual artifacts in the image that I've circled in red on the center dog. If you look in those places on the left dog, you'll see some strange lines or creases in the rendered image. In the right dog, I've welded the shapes together where the visual artifacts occurred. This allows the Wrinkle-Free engine to do its job. If you look in the same problem areas, you won't see any of those line artifacts. They've been all smoothed out by the Wrinkle-Free engine.

I'll be taking a small break from the Wrinkle-Free engine and working on other aspects of Omber for the next month. I may have to finally figure out how to get people to actually use Omber and how to monetize it.