fasterthanlime

Jan 19, 2015 12 min #ooc

ooc generics and flawed designs

ooc is perhaps one of my proudest achievements, but at the same time it's one of the most annoying thorns in my side.

The main reason is that its design is flawed, and some things can't be easily fixed at this point. Now don't get me wrong: every design is flawed to some extent. Design, either when done by a lone coder, or by a committee, never comes out "perfect" — ignoring the fact there is no universal/objective measure of "perfectness".

Concerning ooc, some subjects come up over and over, like interfaces and covers. Part of the issue is that people generally assume that features in ooc are directly mapped from another language and should be identical in every way. But that's not the case. If it were, ooc would probably just be an inferior implementation of C++, or Java, or Ruby. But it's not. It's an inferior implementation of bits of all the above, that I somehow tried to make fit together, along with the contributors that came to help for the past six years.

Here's the thing about generics in ooc: they're pointers to an unknown type, with value semantics.

Let's take a simple generic function:

add: func <T> (a, b: T) -> T {
  a + b
}

Now, if ooc generics were templates, this code wouldn't be any problem at all - it would generate a type-specific version of add, err if the types of a and b weren't exactly the same / compatible, err if there was no + operator for T and T, and so on.

But that's not how ooc generics work. When you have a generic value, the only thing that you know about it is, at runtime:

• Where it resides in memory (pointer address)
• Which type it is (name + size + pointerSize)

But at compile-time, you know nothing. Nothing at all. At compile-time, the only things you can do with generics is "copy them somewhere". And that's just enough for collections, most of the time. ArrayList<T>, HashMap<K, V> etc. don't need to know much about the types they contain.

Well, I lied. They still need to know a bit. Mostly it's about comparison and hashing. For ArrayList<T> indexOf(value: T) -> Int, we need to able to compare two T, otherwise we can't find it in the array.

For HashMap<K, V> we need to able to hash K otherwise we can't put values in buckets, nor compute the hashes of keys when we need to look them up in the right bucket, and so on. And to handle hash collisions, we also need to be able to compare the K type.

How is that done in the ooc sdk? By cheating. Remember how I said the only thing we knew, at runtime, was the address of a generic value, and its type? Well, we can write things like that:

add: func <T> (a, b: T) -> T {
  match T {
    case Int =>
      a as Int + b as Int
    case =>
      Exception new("Don't know how to add type #{T name}") throw()
  }
}

Now, many people out there would think that this is terrible design, that I shouldn't even dare call these "generics", etc. but that's the way it works. There's no use for me trying to argue that 6-years-ago-me made the best choice for how generics worked, I'm just trying to explain how it currently does work so that there's no further confusion.

So back to HashMap, when you build a HashMap, it has a similar match clause on the K type and chooses hashing functions, depending on whether the K type is a string type, a numeric type, or something else.

Now, in a particular issue on GitHub, someone remarked that doing with a function definition like this:

identity: func <T> (t: T) -> T { t }

Then this code worked:

identity(42)

But this code didn't work:

identity<Int>(42)

Whereas if you define a generic class:

Cell: class <T> {
  t: T
  init: func {}
}

Then you could totally do that:

cell := Cell<Int> new()

And as usual, in this flawed but somewhat-internally-consistent design, there's a reason for that.

In the Cell case, the constructor doesn't take an instance of T - so there's no way to infer what T is from the constructor call, so it has to be specified between brackets. Generic types can have type parameters - that's how it works.

Functions are different. In the identity case, if the inferred type (for example, SSizeT) is not what you want, you can always cast the incoming argument:

identity(42)        // -> SSizeT
identity(42 as Int) // -> Int

Now there are some cases where there's simply no way to infer one of the type parameters of a function from its arguments at compile-time, for example:

getSome: func <T> -> T {
  match T {
    case Int =>
      4 // guaranteed by fair dice roll
    case Float =>
      0.8
    case =>
      raise("Can't get some of #{T name}")
  }
}

There's no way that function can ever be called properly. In other languages, you could just do getSome<Int>() or getSome<Float>(), but in ooc you can't. Instead, you can explicitly make T a part of the argument list, like so:

getSome: func <T> (T: Class) -> T {
  match T {
    // etc ...
  }
}

And then you can call getSome(Int) or getSome(Float). Again, one could argue endlessly which is the better approach, the rest of the world seems to have settled on "it's okay to pass types between brackets and arguments between parenthesis" and 6-years-ago-me is stuck on the notion that after all types are just values like any other value and there's no need to have all sorts of constraints around them.

One point for 6-years-ago-me's approach is that if we had a working partial primitive, we could turn getSome(Int) into a getSomeInt easily, and pass that function somewhere else that expects a function that returns an Int:

// fictional code (6-years-ago-me way)
getSome: func <T> (T: Class) -> T { /* see above */ }
getSomeInt := partial(getSome, Int)

eng := GameEngine new()
eng setRandomNumberGenerator(getSomeInt)

Whereas with the usual, use-brackets-for-types style, what could you do? Well, not much apart from having to define your own function, either ahead of time or as a closure:

// fictional code (rest-of-the-world-way)
getSome: func <T> (T: Class) -> T { /* see above */ }

eng := GameEngine new()
eng setRandomNumberGenerator(|| getSome<Int>())

So you lose some higher-order function tools. Which, again, in present-day ooc land, isn't a big deal because the SDK doesn't have partial.

As I mentioned earlier, it's really quite hard to do anything useful with generic values. That is, until you coerce them back to a 'real' value.

someFunction: func <T> (t: T) {
  // `t` is kind of useless
  u := t as Int
  // Now, with `u` we can do anything we want.
}

Coercing a generic value back to a real/simple value is a dangerous operation. If the type is wrong, it could lead to garbage data - the compiler doesn't do any check there, because it trust you to know what you're doing (which, in retrospect, might be a mistake). But in 6-years-ago-me's defense, there's quite a nice mechanism to coerce generic values back to real/simple values in a relatively-safe way:

someFunction: func <T> (t: T) {
  match t {
    case u: Int =>
      // we're sure `t` was an Int and now we can use it as u
    case =>
      // error handling goes here.
  }
}

So, there. Those are the limits of how generics work. To add to the defense case, the inference engine is pretty smart (considering..), to allow you to omit as many types as possible.

For example, something like this works within the confines of the aforedescribed system:

Sorter: class <T> {
  compare: Func (T, T) -> Int
  init: func (=compare) {}
  sort: func (l: List<T>) -> List<T> { /* ... */ }
}

s := Sorter<Int>(|a, b| (a < b) ? -1 : (a > b ? 1 : 0))
s sort([1, 2, 3] as ArrayList<Int>)

So now, whenever someone comes along with misconceptions about generics, covers, interfaces, and strong opinions on how they should work because "it works that way in language X or Y and the feature has the same name", then I don't know what to respond.

Sometimes they're legit bugs that can be fixed, and when it's the case, I happily help in the limits of my free time. But sometimes, it's just not possible. Often, it's a choice between:

• Making a 1% case work, "by chance", given 12 conditions that people have to know about (until the next person that stumbles upon any of the other 99% and complains about it - righteously!), by adding another patch to the compiler on top of dozens of other patches
• Take the high road, have a big discussion about how a feature should work, rewrite 10%, 20%, 50% of the compiler (6-years-ago-me had no idea how to make a truly modular compiler, but still crammed a lot of things in there - impressive then, annoying now), and lose compatibility with 80% of the ooc code ever written (which isn't much but still a few hundred thousand lines of code, some of it still being used by people day-to-day, myself included)

The first option, I took too often. It doesn't work. When you have a good, solid design, patches add the polish you need. When you have a flawed design, like cursed 6-years-ago-me came up with, adding patches just adds crust, and technical debt, and even more user frustration down the road.

It's like if you have a bug in a game "non-flat surfaces can't be climbed by player". And someone comes up and says "hey, in world 3, the stairs in the old mill behind the lake, I can't climb them. Here's a patch." And the patch is that whoever is standing at the bottom of the stairs, connected from an IP starting in 174.*, who holds Ctrl Shift and Alt and looks up, is immediately teleported to the top of the stairs. Sure, "there's no way to walk down the stairs yet, but it'll come in the next patch!"

Except there's no next patch, because by then the person has realized that the problem is much larger than just the stairs in the old mill behind the lake. It's that only flat surfaces that work well. Since they don't have the time to rewrite an entire game engine that doesn't assume that every surface is flat, they move on to another game. And if their patch has been merged, it just rots there forever, because chances are nobody will think to press Ctrl, Shift, Alt, and look up at that particular spot.

On your side, the game dev, sure, you could document that the game ONLY works for flat surfaces. But maybe you don't want to because you're a bit ashamed and you don't want to go on record saying that you didn't have the time/energy (other will think "skill") to get non-flat surfaces to work properly. After all, you're supposed to be an expert, you should know better.

So you don't put it on the front page of your game's website, because that would just be bad publicity. There's a fine line between outright lying and just shooting yourself in the foot. Also, you've moved to another game project, but you still go back to that particular game from time to time and play it and add other things to it, so it's not really abandoned but not really your main focus either - and it's cool with the long-time players because they know the deal and they're already having lots of fun with flat surfaces only.

I'm going to stop until the metaphor drags on forever, but hopefully you get the point. When you have to deal with a flawed system (and everything in programming is, to some degree), sometimes it's worth it to try and change it for the better, but most of the times it's best to see if you can work within the limits that this flawed system has and still get to achieve your goals. If you don't, then maybe it's simpler just to switch to another flawed system that suits you better (and then later on be frustrated because that other flawed system doesn't have some features that you liked in the first one... and so on).

This kind of shit used to keep me up at night: but how do I make it perfect? So that it suits everyone's use cases? And that the code is beautiful? And fast? And that I can code before life exits my poor, wasted-youth human?

But it doesn't anymore (not as much anyway), because nowadays I'm happy just combining flawed systems to do what I want, and I try to throw a little poetry in there to keep the artistic side of my being happy. Sometimes it's unicode snowmen, sometimes it's quirky loading messages. It's childish, and this post is probably a cop-out, and I'll still sigh when I see someone who wishes they could change half of ooc for the better, but I don't think it's going to happen. That would turn it into a C++ or a Scala, and I really don't want to be held responsible for that.

Comment on /r/fasterthanlime

Here's another article just for you:

The bottom emoji breaks rust-analyzer

Feb 13, 2023

1 hour 10 min #rust · #emacs

Some bugs are merely fun. Others are simply delicious!

Today's pick is the latter.

Reproducing the issue, part 1

(It may be tempting to skip that section, but reproducing an issue is an important part of figuring it out, so.)

I've never used Emacs before, so let's install it. I do most of my computing on an era-appropriate Ubuntu, today it's Ubuntu 22.10, so I just need to:

Read more