typeck hang with recursive type

[jruderman]

This hangs the compiler in typeck:

enum t = @t;
fn main() {
    let x : t;
    x.next = x;
}

[nikomatsakis]

probably my fault.

[nikomatsakis]

Not at all what I expected it to be. Rather, a loop in autoderef. I will fix the loop but I suppose it might also make sense to reject this type as uninstanitable---there should be some enum variant that does not require an instance of the enum itself. But I am not sure this is worth checking in general: after all, the user could never successfully construct an instance of that type, and to check it in general requires expanding and traversing enums and so forth.

[jruderman]

I think rejecting the type makes more sense. If we reject the type immediately, we won't need to change the autoderef loop, right?

[nikomatsakis]

That's the idea. I independently came to the same conclusion. In particular, there are multiple autoderef loops (which ought to be consolidated more, but still) and I don't like the idea of this same kind of bug creeping up in another of them.

[nikomatsakis]

Also, it's just a nicer error message.

[nikomatsakis]

Unfortunately, we still need the loop check in the autoderef loop for typeck, because the actual checking that the enum is instantiable and the checking of the fn body occur in the same pass over the AST. We could change it to have three passes (one to compute types and generate info, one to check for instantiability, and one to check fn bodies and so forth) but that seems annoying.

[jruderman]

Could the pass bail out early after encountering an uninstantiable type?

[nikomatsakis]

@jruderman No, the problem is that the autoderef loop could occur before we get around to checking whether the type is instantiable. As I said, I could restructure the pass to make sure we check all types for instantiability (and possibly other properties) before we delve into function bodies and expressions, but otherwise I think we still need to be careful when autoderef'ing. Such a restructuring might be a good idea though.

[nikomatsakis]

Fixed in 23f92ea

[msullivan]

Should this be closed, then?