Variable declaration vs assignment

[akva]

I figured that general discussion of CoffeeScript belongs to issue tracker. So here it goes.

It seems like coffee-script does not distinguish variable declaration and assignment. This means that, unless you are very careful with naming local variables, it's easy to unintentionally assign to a global variable. Consider this js code:

sys = require('sys');
var foo = 42;
(function() {
    var foo = 43
}());
sys.puts(foo);

Here variable foo inside a function is local to that function. It is not possible to reproduce this code in CoffeeScript. CoffeeScript 'equivalent' will override global variable foo.

sys = require 'sys'
foo = 42
(() -> foo = 43)()
sys.puts foo

This may lead to hard to find bugs. What's your thoughts on this?

Regards,
-- Vitali

[StanAngeloff]

It has been well documented and discussed on the tracker before. I am actually quite happy with how Coffee handles scoping at the moment.

[akva]

I just read the documentation again more carefully. Indeed, I was too quick to post an issue. My bad. However, let me elaborate on this issue a little bit. This is a part of the documentation that explains my point in fewer words.

Because you don't have direct access to the var keyword, it's impossible to shadow an outer variable on purpose, you may only refer to it. So be careful that you're not reusing the name of an external variable accidentally, if you're writing a deeply nested function.

Consider this code

...
... many lines of code (likely not written by me)
...
func = () -> # a new function that I add
    foo = 42 # not clear if I declare a local var or assign a global one

JS has different syntax for variable declaration var foo = 42 and assignment foo = 42. The problem with it imho is that declaration syntax is more verbose than assignment syntax (and var is easy to forget). It seems that variable declarations, especially if one tends to write in functional style, appear in source code more often than assignments. Therefore declaration syntax should be more concise and assignment could be more verbose.

My preferred syntax would look something like this:

foo = 'foo'
bar = 'bar'
...
... many lines of code
...
func = () ->
    foo = 42  # declares local var. shadows global
    bar := 43 # assigns global var
    fooo := 44 # compile error

This approach is only slightly more verbose but makes programs more explicit and robust, imho. And again, in my own code I tend not to use destructive assignment that much.

What do you think?

P.S. Python uses nonlocal keyword but it's a little overkill for my taste.

[StanAngeloff]

How about referencing a variable from the global scope? If you simply do alert bar from a function, should it throw an exception if bar is not in scope? It seems like it should since assigning does so and we need things to be consistent.

When you are writing in Coffee, you will not always have full control over what files you interface with. While it may be possible to throw compile time errors when referencing a non-existing global variable, we cannot reliably do so when you are working on a file that just happens to be a small part of an application which in turn might have been written in pure JavaScript.

Coffee is also used on the server-side. In node.js it's common to have many callbacks, like so:

fs.readFile 'file.js', 'utf8', (err, contents) ->
  # do some other stuff
  process.nextTick ->
    throw err if err

Here we are referencing err in a function that is nested inside another function. Since err is not in scope, but it's also not part of the global one, how do you assign or reference it?

Keep in mind Coffee is a dynamic language just as JavaScript is. There is no type checking, no NullReference exceptions and no actual compilation.

If I have to point out one benefit of the existing scoping, it would be that it's consistent. Define once and use everywhere.

[akva]

Maybe I didn't express myself clear enough. I guess I should have used the word outer scope instead of global.

I am not proposing to change the scoping rules. Quite the opposite, I propose to keep them exactly the same as in JavaScript, and therefore making Coffee more consistent with JavaScript (I am as big fan of consistency). One way to do it is to introduce the following syntax

1. foo = 42 to mean JavaScript's var foo = 42;
2. foo := 42 to mean JavaScript's foo = 42;

Right now foo = 42 can mean two different things in Coffee depending on the context. It can be a variable declaration, or an assignment to a lexically scoped variable. And this is not consistent.

So your example with fs.readFile will just work unchanged as expected. err is in lexical scope of the nested function so everything is all right.

[goj]

How about keeping current semantics and re-introducing JavaScript's var keyword to force declaration?

v = 1
fun = () ->
    var v = 2
fun()
alert v # shows 1

[akva]

If I understand correctly what you mean by forced declaration, it doesn't really solve the issue. v = 2 is still ambiguous (can be local declaration or assignment to outer scope) depending on the context, and the context is as big as the whole source file. What "forced" declaration says is - I know there is this variable v in the outer scope and I want to shadow it. What I am trying to achieve is when I see v = 2, I could conclude - This is a local var and I don't need to check whether it's already declared in outer scope or not

To give anouther example. Say you have a function somewhere at the bottom of the source file.

fun = ->
    v = 2

Here v is a local var. Fine. Then a few months later a new developer comes in and adds v = 1 at the top of the file. Now, he altered the behavior of fun without even touching it.
And this will be hard to debug.

JavaScript copied scoping rules from Scheme but imho it got the defaults wrong. var should have been implicit and assignment more explicit.

I realize that proposed foo := 42 syntax might look old fashioned. I don't insist on this particular one. It's just the most concise form I could think of.

[goj]

Yes, you understood me correctly.

AFAIK there is no way to shadow variables in CoffeeScript, which
is, as you explained is a big problem.

Using var was a proposal to allow that in backward-compiatible way.
This is much better than only solution that we have today
("be careful with naming your variables") but it results in almost copying
the problem JavaScript have today
("always remember to put vars in your code or you'll be in trouble").

The more I think about your proposal the more I like it.
It really solves the problem and gives us more consistency with JavaScript.

It is a huge backward incompatibility, but hey, CoffeeScript is before 1.0 version
and this really seems worth it.

Speaking of syntax, := stands out a little, but it should stand out when we change
something from outer scope, so that's more than OK. The other way to go would
be to use some sigil, like &, which would play nice with destructuring
assignment:

[&im_from_outer_scope, declare_me] = stuff

Other way to word akva's proposal that would probably make Stan happier is:
you assign to local variable using = (as you do now) but if you want
to change something from the outer scope, then you have be explicit
(use :=). This makes both code easier to read and bugs harder to introduce.

[satyr]

Note that [we do have a way](http://satyr.github.com/cup/#foo%20=%2042%0A((foo%29%20-%3E%20foo%20=%2043%29(%29%0Aputs%20foo) (+1 by the way) to shadow outer variables.

See also: #238

[akva]

Speaking of syntax, := stands out a little, but it should stand out when we change something from outer scope, so that's more than OK.

Exactly. Destructive assignment is a dangerous operation and should stand out as a warning. And, to repeat myself, programs written in functional style won't have too many :='s. If one finds oneself using too many :='s it's probably a good sign that code needs refactoring.

you assign to local variable using =

Don't want to sound pedantic but just to make sure we're on the same page. By assignment I mean destructive assignment, or rebinding the earlier declared variable. In this sence you use := to assing/rebind local var too. That is:

local = initialize()
if something()
    local := new_val()

@satyr

Note that we do have a way to shadow outer variables.

Yes, but shadowing is not the problem.
I've been thinking about the best way to summarize my issue and I think it can be summarized into the following two problems:

1. Delocalization (I am just making up the term)
   Every time you indroduce a new variable in scope A you have to make sure the variable name is not already used in any of the nested functions. Otherwise you'll break it.
2. Ambiguity
   var = val is ambigues. To understand what it means you have to inspect all outer scopes.

It is a huge backward incompatibility, but hey, CoffeeScript is before 1.0 version
and this really seems worth it.

Totally agree. It took Python 3.0 to get the scoping right - http://www.python.org/dev/peps/pep-3104/ (though they had a different problem to start with). CoffeeScript has a chance to get it right from the very beginning. Especially considering that it's not hard to implement, since JavaScript already gets the scoping (almost) right.

[jashkenas]

Sorry, folks, but I'm afraid I disagree completely with this line of reasoning -- let me explain why:

Making assignment and declaration two different "things" is a huge mistake. It leads to the unexpected global problem in JavaScript, makes your code more verbose, is a huge source of confusion for beginners who don't understand well what the difference is, and is completely unnecessary in a language. As an existence proof, Ruby gets along just fine without it.

However, if you're not used to having a language without declarations, it seems scary, for the reasons outlined above: "what if someone uses my variable at the top of the file?". In reality, it's not a problem. Only the local variables in the current file can possibly be in scope, and well-factored code has very few variables in the top-level scope -- and they're all things like namespaces and class names, nothing that risks a clash.

And if they do clash, shadowing the variable is the wrong answer. It completely prevents you from making use of the original value for the remainder of the current scope. Shadowing doesn't fit well in languages with closures-by-default ... if you've closed over that variable, then you should always be able to refer to it.

The real solution to this is to keep your top-level scopes clean, and be aware of what's in your lexical scope. If you're creating a variable that's actually a different thing, you should give it a different name.

Closing as a wontfix, but this conversation is good to have on the record.

[akva]

I was going to write a long response, but instead decided to replace it with a short question. Here

foo = 42
(-> for foo in [1..2] then)()
alert foo # 42

loop variable foo (which is just a local variable?) shadows the top-level foo, which is inconsistent with the rest of the language. How do you explain this inconsistency?

[jashkenas]

I'm not saying that shadowing isn't technically possible -- just that it's bad style, and CoffeeScript shouldn't make it easy for you to accomplish. It would be far better to use a different name for that index.

[akva]

I totally agree with you that shadowing is a bad style. And instead of shadowing it's indeed better to just choose a different name.

But I am not talking about deliberate shadowing. I am talking about situations when you are not aware that top-level variable exists and assign it accidentally, or when top-level variable is added later. Check this example from my comment. The body of fibonacci function is supposed to be a black-box, but it's not. And what's worse, even unit test won't spot it - when run in isolation fibonacci function will work just fine, but will break in the context of the whole module. In other words the current scoping rules break encapsulation. This example is particularly illustrative since cache is such a common name and can easily clash.

[satyr]

 foo  = 'foo'
 bar  = 'bar'
 func = ->
   foo   = 42  # declares local var. shadows global
   bar  := 43  # assigns global var
   fooo := 44  # compile error

Adopted this strategy on my coco branch and am quite happy with the decision.

[akva]

@satyr Looks great. Would you like to share some more details about your experience of using it?

Honestly, I reckon Ruby scoping rules are just wrong. It seems that they may change in Ruby 2.0