The 64 bit question

This question has been in the back of my mind for a few months now. So far, REBOL 3.0 still uses 32 bit integers in order to get it up and running as quickly as possible.

However, the time is coming over the next couple weeks to make the big jump to 64 bits. The advantage of course is that integers can hold much larger values, such as the sizes of very large files, etc. The disadvantage is that 64 bit values require more CPU time for memory access and math operations. They also take twice the memory space, although REBOL already allocates that space, so most programs won't see that increase.

The 64 bit change could also affect more than integer values. For example, do we want to allow series (e.g. strings, binaries) to be larger than 2/4GB? I'm not talking here about files (see below); I'm talking about actual in-memory series. If we want that, not only does the management structure (e.g. tail position) increase, but so does the series index pointer (to be able to address extra-large series). Although this is not such a big deal for the structure definitions, it is a big deal to the run-time code; REBOL is packed full with references to things like tail and index (for bounds checking, etc.).

And finally, there is the idea that perhaps we could use a compile-time switch to build both 32 bit and 64 bit versions. A 32 bit version makes more sense for things like cell phones and PDAs. But, it raises the issue that these versions of REBOL would be closely compatible but not perfectly compatible. Perhaps that could be improved by using 64 bit integers, but 32 bit series.

As with any design process, there is a range of tradeoffs to consider. Certainly, the needs of REBOL users should help us to determine the best mix between the choices. Personally, I like the idea of 64 bit integer math, but the 32 bit series restriction is fine with me. I would be using seek to address such large files anyway.

And, with regard to files, I should note that I am planning that the new port! datatype will allow 64 bit index access. The new ports will no longer store their index positions within their references, but within the port object itself. The port system will work more like traditional file access than it has in the past.

36 Comments