I agree there was a case for raising this. I'm sorry if I gave any other impression. ( I don't know why u raised this with me only and not the list? But I have)
But, I don't think the case is a valid one, because you are finding difficulty with a dynamic language. Dynamic languages offer power. With power comes responsibility. Responsibility in my book equals things like comprehensive UT. If you don't test you fail.... please demonstrate another industry that doesn't consider testing an essential element of what they do?
- This particular one is a hobby project, and I don't really fancy spending my free time writing unit tests.
I'm sorry to be contrary... but... if you are a hobbyist... should you really be commenting on the semantic or operational aspects of a language?.... if you are not writing unit tests then you are not behaving like a professional... if you are not, then why should a language implementation adhere to you? If you think that not being a professional excuses you from doing a good job then that is equally worrying.... UT is not a pro Vs hobbyist choice... its how one builds good software... QED... if u can't be bothered then what r u doing?
I hate to bang the drum.. but I am tired of silly crap....
- Unit tests, by their nature, don't cover everything. What if the unit tests never passed in a value beginning with "-"?
Then they are crap unit tests (and probably crap code).. if failed to expect 'dirty' inputs....
I fail to see how someone who admits to not being professional can then create a justification for a language change..... I'm more surprised anyone's taking it seriously!
Regards
Simon
On 4 Aug 2010, at 18:43, Kristoffer Lawson wrote:
On 4 Aug 2010, at 19:50, SImon Millward wrote:
Isn't it fair to say that by design, any dynamic language has such possible traps and pitfalls... (although this one is especially awkward). You don't need to try very hard to make TCL fall over in all sorts of ways like this.
Absolutely, dynamic languages do have these (and static languages do too, just in a different way). However I think for many things Tcl is probably reasonably OK. PHP is notorious for holes.
But yes, this a particularly bad one because one normally assumes you can send variables as parameters to methods and constructors quite freely. It's common to be initialising with random pieces of string, for various purposes. It's obvious that something like [eval] or [after] is going to be dangerous. Perhaps even [switch] (although I think there'd be a case against that too). But creating a new ob with initial data passed as a variable?
The question I always ask my teams when something like this comes up is
- Why didn't the unit tests catch it
- What 'stock' unit test will we add in future to catch it
This particular one is a hobby project, and I don't really fancy spending my free time writing unit tests.
Unit tests, by their nature, don't cover everything. What if the unit tests never passed in a value beginning with "-"?
The flipside to this, as mentioned, is that the resolution is actually quite ugly, and would need to be utilised for safety reasons almost every time a class is instantiated. I think there is definitely a case for raising some concern over this.
-- Kristoffer Lawson, Co-Founder, Scred // http://www.scred.com/
On 5 Aug 2010, at 02:05, SImon Millward wrote:
I agree there was a case for raising this. I'm sorry if I gave any other impression. ( I don't know why u raised this with me only and not the list? But I have)
My mistake :-) Not hitting the "Reply to All"...
But, I don't think the case is a valid one, because you are finding difficulty with a dynamic language. Dynamic languages offer power. With power comes responsibility. Responsibility in my book equals things like comprehensive UT. If you don't test you fail.... please demonstrate another industry that doesn't consider testing an essential element of what they do?
I think we're entering a totally unrelated debate here. For what it's worth, I've been using dynamic languages for over 15 years (Tcl in particular), so it's not like they're something totally new to me. This particular problem does not, for instance, exist in Python, whereas PHP is riddled with many (and, in practical situations, has often caused problems for people). I'd say the basic rule should be that common usage with argument passing should be safe. It should not be necessary to do every single instantiation like this:
set ob [MyClass new [list -init arg1 arg2 arg3]]
This makes common use awkward and something that will easily trip up users. Consider that in an OO environment, object instantiation is one of the things you will be doing a lot of. Keep in mind you have to do the above every time, just to avoid mistakingly sending something which will cause a crash or a hole. It seems Gustav agrees, as he pointed out this will not be the case in 2.0.
- This particular one is a hobby project, and I don't really fancy spending my free time writing unit tests.
I'm sorry to be contrary... but... if you are a hobbyist... should you really be commenting on the semantic or operational aspects of a language?
Uhm.. Yes? I didn't realise this community was closed off to hobbyists. I imagine many people using Tcl or XOTcl fit that bill. Naturally it's the right of the authors to implement their language however they feel — it's lot like I'll bring them to court (plus I've enjoyed beers with one of them :-) — but it is also my right not to be happy about a particular implementation.
- Unit tests, by their nature, don't cover everything. What if the unit tests never passed in a value beginning with "-"?
Then they are crap unit tests (and probably crap code).. if failed to expect 'dirty' inputs....
Well, I raise my cap if your unit tests are always 100% great, with 100% coverage of all possible values and you have code without bugs. We should probably hire you.
I fail to see how someone who admits to not being professional can then create a justification for a language change..... I'm more surprised anyone's taking it seriously!
I'm talking about common sense here. A feature of the language makes it awkward to write safe code, and leads to error prone results. For something so basic as object instantiation, requiring awkward syntax is just not going to go down well with many coders. It reduces readability and, in fact, none of the XOTcl examples you see use the safe instantiation model.
There are a few gotchas like this in Tcl. [switch] is dangerous, but the "--" pattern is well established (although I'm not a big fan of that). [puts] is something I imagine most code has got wrong, but the consequences are generally not particularly dangerous. At the end of the day programming languages are for humans, and need to consider that as a factor.
I don't know why I'm ruffling your feathers so much. It's true my initial posting on this was filled with late night "Argh", but I stand by the point. Also, I've been using on and off XOTcl since the 0.8x days and have always been a fan of the language. I've done my part to evangelise it. That it has taken this long to stumble across this issue is exactly why it is particularly dangerous.
Dear all,
In my understanding, Kristoffers point was that although he is using XOTcl since many years (i see postings from him more than 10 years ago) and although he is a experienced programmer, he was not aware of the problem with argument values starting with a dash (certainly not all, e.g. negative numbers are fine), and he was not aware of the escape mechanism (which was added to XOTcl early 2003, see Changelog).
The primarily problem is external input, unfortunately, one weak element of most scripting languages. Similar problems lurk around in Tcl in scripts like
set file1 -force set file2 -violence file delete $file1 $file2
where one should certainly use "--" to avoid the problem ... once the developer is are aware of the danger. If a developer is not aware it, the tests he is writing are most likely not covering it properly. But a developer frequently writing tests is most likely aware it.
Btw, google codesearch returns more than 5 times more hits for "file delete $" than for "file delete --". It is not unlikely, that there are many such problem lurking around in may tcl scripts. The problem is not only "file delete" but as well "file copy", "file rename, any many other tcl commands supporting "--" (switch, lsearch, ...). And the problem is not only about flags starting with a dash (just syntactic sugar), but as well with all other cases, where the interpretation of a <value> depends on an interpretation of a <keyword>, especially, when the <keyword> is optional. In Tcl, one cannot distinguish between "-force" passed as a literal or "-force" passed via variable. If we could, life would be in this respect easier. Compiled languages have it easier, when they require that the <keywords> are available at compile time.
The case with the dash arguments in XOTcl is especially nasty to the unaware, since the standard configure method determines on the "-" (a) where the last argument list ends, and (b), what to call as a method. If user-data is passed to the object creation, using the list notation is therefore a must (as mentioned by Artur)
People fully aware of the XOTcl flexibility have no big problem with this, since they can write their own "configure" method (this method implements the dash interpretation) and use this (e.g. via mixins) where appropriate.
-gustaf neumann
On 5 Aug 2010, at 10:59, Gustaf Neumann wrote:
In my understanding, Kristoffers point was that although he is using XOTcl since many years (i see postings from him more than 10 years ago) and although he is a experienced programmer, he was not aware of the problem with argument values starting with a dash (certainly not all, e.g. negative numbers are fine), and he was not aware of the escape mechanism (which was added to XOTcl early 2003, see Changelog).
Cheers, and apologies if any offense was caused by my postings. This just caught me completely off guard. Of course once I saw it, it was obvious what is happening, but it really is something you can easily miss, especially as existing XOTcl code out there will not generally worry about it.
The primarily problem is external input, unfortunately, one weak element of most scripting languages. Similar problems lurk around in Tcl in scripts like
set file1 -force set file2 -violence file delete $file1 $file2
Indeed. The slight saving grace for those cases is that, in practise, one tries to look more carefully before operating on any filenames passed by the user, but I agree it's quite easy to get caught out there too. Maybe Tcl should have used a different model for optional arguments (something like [file {delete -force} $file] perhaps?), but it's unfortunately difficult to turn back the clock on that...
The case with the dash arguments in XOTcl is especially nasty to the unaware, since the standard configure method determines on the "-" (a) where the last argument list ends, and (b), what to call as a method. If user-data is passed to the object creation, using the list notation is therefore a must (as mentioned by Artur)
Yes, this is also true for setting a parameter:
Car new -doors $amount
If $amount contains a dash, it'll do a method call. Just something to be aware of. It's also not just for user-passed values either. The same could apply to random data, or binary data, or the contents of a file, for instance. Watch out if you're doing a YAML parser.
People fully aware of the XOTcl flexibility have no big problem with this, since they can write their own "configure" method (this method implements the dash interpretation) and use this (e.g. via mixins) where appropriate.
Yes, if something positive has to be said then it is that flexibility. I went one further and already created a method directly into 'Class' ;-)
Am 05.08.10 10:38, schrieb Kristoffer Lawson:
Yes, this is also true for setting a parameter:
Car new -doors $amount
If $amount contains a dash, it'll do a method call.
Well, "contains" is too strong, "starts with a dash followed by alpha" is precise.
With XOTcl 2.0 (as now in git) the following cases work as expected
package req XOTcl 2.0; namespace import ::xotcl::*
Class Car -parameter {{doors 4}} Car c1 -doors -2 Car c2 -doors a-b Car c3 -doors -b
with XOTcl 0|1.*, case "c3" was leading to the problem.
With XOTcl 2.0, one would most probably use
Class Car -parameter {{doors:integer 4}} Car c1 -doors -2 Car c2 -doors a-b
to trigger an error on "c2", or better define your own checker "posint"
Class Car -parameter {{doors:posint 4}}
to get already errors on c1.
-gustaf
On 5 Aug 2010, at 12:11, Gustaf Neumann wrote:
Am 05.08.10 10:38, schrieb Kristoffer Lawson:
Yes, this is also true for setting a parameter:
Car new -doors $amount
If $amount contains a dash, it'll do a method call.
Well, "contains" is too strong, "starts with a dash followed by alpha" is precise.
Sorry, yes. I need to check still what 2.0 does with:
Car c1 $a
where $a starts with a dash.
Class Car -parameter {{doors:integer 4}} Car c1 -doors -2 Car c2 -doors a-b
to trigger an error on "c2", or better define your own checker "posint"
Class Car -parameter {{doors:posint 4}}to get already errors on c1.
I think these are fine. Errors when given a parameter of the wrong type are totally expected, and cannot be abused to execute code with bad values.
Am 05.08.10 12:21, schrieb Kristoffer Lawson:
I need to check still what 2.0 does with:
Car c1 $a
where $a starts with a dash.
The next release comes with two flavors: (a) XOTcl 2.0, which is mostly backward compatible, (b) a new object system with a different syntax (among many other things, one cannot call arbitrary methods via dash syntax).
For XOTcl 2.0, the behavior for this example is like in XOTcl 1.* (it has to be for compatibility for many one-liners out there), for the new syntax, you get an error message (... and, by default, one has to use "Car create c1" instead of "Car c1", since the latter is dangerous as well).
-gn
On 5 Aug 2010, at 13:38, Gustaf Neumann wrote:
For XOTcl 2.0, the behavior for this example is like in XOTcl 1.* (it has to be for compatibility for many one-liners out there), for the new syntax, you get an error message (... and, by default, one has to use "Car create c1" instead of "Car c1", since the latter is dangerous as well).
Yeah, I agree the latter is dangerous, if in a different way. Basically you can end up with really obscure bugs if you mistype "instproc" somewhere. I speak with experience :-)
Must get round to reading your papers on XOTcl 2.0. They're open and waiting for me to get round to them...
I resolved my own situation with sending args with "-" at the start by replacing stuff for Class. I then changed that to use a mixin, as per Gustaf's suggestion. I wanted to do this for everything without having to bring in the mixin to every class, or creating a new meta-class, but of course a couple of external libs don't work as they use the dash parameters. This got me thinking. I do see that it is a handy notion to have sometimes, so played around with several ideas on how to do that.
I'm thinking aloud here, so excuse me for rambling.
Anyway my idea is to use a different method, instead of [new], for the cases where pre-configuration is desired. Here are some of the options:
IrcConnection preconf {-nick $IrcNick -channel $IrcChannel} $IrcServer
IrcConnection -new {-nick $IrcNick -channel $IrcChannel} $IrcServer $IrcChannel
IrcConnection -new $IrcServer $IrcChannel {-nick $IrcNick -channel $IrcChannel}
IrcConnection -new $IrcServer $IrcChannel { -nick $IrcNick -channel $IrcChannel }
IrcConnection -new $IrcServer $IrcChannel { nick $IrcNick channel $IrcChannel }
Of these I actually like the last one most. It's almost as if you have a Tcl script passed in. In fact, why not implement exactly that way, but with implicit access to the object's methods in-scope. It also solves the problem of passing values beginning with dash as arguments to the parameters. To use without configuration, just use the normal [new] method, or have an empty configuration script.
One option would also be to have the last option, but with the normal [new] command. It would work so that any mandatory arguments to the constructor are always picked up first (and only the last argument is configuration). Naturally this doesn't make those cases safe where 'args' or optional arguments are used, so would not be 100% safe.
Apologies for the flood, but I can't get my mind off this. Some more ideas for how this could be made to work:
== Use a brand new command:
set ob [new Car "Land Rover"]
set ob [new -with {-doors 2} Car "Caterham"]
or
set ob [new -with { doors 2 } Car "Caterham"]
Here there is no real risk of pre-initialisation getting confused with arguments to the constructor. It doesn't look quite as nice as my earlier suggestion:
set ob [Car -new "Caterham" { doors 2 }]
But it would mean not having to have both [new] and [-new]. The current [new] method could then be deprecated, without breaking old scripts.
== Do something a bit like Objective C:
set ob [[Car create] init "Land Rover"]
set ob [[[Car create] conf -doors 2] init "Caterham"]
or possibly:
set ob [Car create] $ob doors 2 $ob init "Caterham"
This could probably be done in a near backwards compatible way. It's very explicit without any funny logic. In fact you can almost do it right now, except there is no method to create with an autoname, but to not do argument configuration and construction calling.
One more cheeky idea, which might be too hacky. You can actually check what was used to call a method (see [info frame]). If the dash values were passed as-literals, then handle them as options, otherwise as values. This is probably quite un-Tclish, and possibly confusing, but I believe it is possible.
-
Gustaf Neumann -
Kristoffer Lawson -
SImon Millward