There are three things I've been thinking a lot about recently: module systems, build systems, and type systems. These disparate technologies are related in funny ways: a build system inherently falls out of a module system, and type systems must be aware of module boundaries. I mainly want to talk about build systems and how I plan to develop my project (some talk of type systems will probably creep in).
Right now, I'm handwriting Makefiles (for GNU make). I wonder how many people just cringed. My project is really small right now, so it's suitable. It's also really easy to write the Makefile, because I don't specify dependencies.
You see, I'm using a language called Scheme, which is on the level of Python in terms of "higher-level"-ness. I don't really even need a build system, since I could load up all of the code at runtime and interpret it. However, I'm using an implementation of Scheme called Gambit-C which compiles to C code, and ends up running really fast when compiled. So I want some way of compiling code automatically.*
It turns out that it's really easy as long as I stick with Gambit-C Scheme for the whole project. I just need to use the procedure (LOAD name). At runtime, LOAD looks for the compiled dynamic library under that name, loads it if it finds it, and otherwise tries to load in the source code file for interpretation. I write my project using individual scheme (.scm) files as modules, and load in any dependencies like so:
--- render.scm ---(load "opengl")(load "scene")(define (render-it thing) (glBegin GL_TRIANGLES) ...)Now, I can choose whether or not to compile a module (a scheme file) into a dynamic library. If I don't compile it, Gambit will simply load in the source and evaluate it.
Since the LOAD procedure happens at runtime, if I do compile a module, none of its dependencies are compiled with it. This means that I don't need any special mechanism of recompiling a module when I change one of its dependencies, since it will always be loaded at runtime.
I really like this method. Marc Feeley, author of Gambit-C, obviously wrote LOAD with this kind of usage in mind. It's like compiling individual C files into dynamic libraries and loading them at runtime - but that would be more difficult for a few reasons (I don't think you can reload dynamic libraries, wouldn't that cause duplicate symbols to conflict?).
The problem is that I don't think Gambit-C Scheme is going to cut it performance-wise for my whole project. I plan on developing in it for a while, but eventually at least part of the core system will need to be rewritten in C or PreScheme. When that comes, I'll need to figure out how I want to develop the system. I'm enjoying the flexibility of dynamically loading in libraries, since I can reload them at runtime, compile them separately, etc.
I guess I want to avoid creating a large C codebase that takes hours to compile, but maybe I'm over-analyzing it. After thinking through the big difference it would make, I realized it comes down to this: static typing versus dynamic typing.
C, C++, and even PreScheme are statically typed, meaning the compiler can reason about the type of every piece of data at compile-time. This has a huge impact on the coding style and how its built. Technically, it's the reason why header (.h) files exist. And it's the reason why it's easy for build times to increase as the codebase size increases, even for small changes. Whenever an interface changes, you have to rebuild all modules depending on it (to re-verify all the types).
Scheme, on the other hand, is dynamically typed. There are no assertions about types at compile-time; rather, every piece of data is tagged with a type at runtime. This gives you enormous flexibility, such as being able to do what I described above with LOAD.
I favor both systems. Static typing lets you prove something about the whole program at compile time, which can be a huge time saver when finding bugs. When I use dynamic typing, I tend to write a huge amount of unit tests in an effort to cover every execution path. Static typing also enables many optimization techniques. On the other hand, dynamic typing gives you a lot of flexibility. I've discovered that dynamic typing is fine AS LONG AS you have an appropriate debugger, and I tend to develop faster when using it.
I predict that I'll have a mix when my project gets bigger. I like to prototype with Gambit-C Scheme since it is really fast to develop with. However, for optimized code that needs to be maintained (probably parts of the core libraries), I'll probably end up writing them in a statically-typed, faster language. When this comes, it'll be interesting to figure out what kind of a build system I want to use.
My first couple posts will be thought dumps. I'd like to hear if anyone else is using a higher-level language like Scheme, Python, or something else for high-performance graphics development. Have you enjoyed it, or do you miss C or C++? How do you typically develop (in what IDE, what build process, etc.)?
* I also need to compile scheme files that contain C code. Gambit-C has mechanisms for inlining C code or interfacing into external C code, and this code has to be compiled for obvious reasons.
