On Sun, 23 Oct 2011 18:20:32 +0300, Alex Rønne Petersen <xtzgzorex@gmail.com> wrote:

> On 23-10-2011 14:51, Vladimir Panteleev wrote:
>> On Sun, 23 Oct 2011 14:06:42 +0300, Timon Gehr <timon.gehr@gmx.ch> wrote:
>>
>>>> I don't see how my generateReflectionForModule idea requires any
>>>> modification of the reflected code. The module can generate RTTI for all
>>>> types used recursively from the starting point. If more precision is
>>>> needed, it can be specified as generateReflectionForModule/etc.
>>>> parameters.
>>>>
>>>
>>> The module can generate RTTI for all types recursively from the
>>> starting point iff that information is statically available. It does
>>> not have to be. A module that comes as .di + static library binary
>>> could return a reference to a private class that has a publicly
>>> exported base class. How would you generate RTTI for a statically
>>> invisible class?
>>
>> The only way to prevent such problems is to have RTTI generated by the
>> compiler and enabled always or by default.
>>
>> However, is such functionality even desirable? It is equivalent to
>> having a back door for accessing private class members (implementation
>> details in general).
>>
>
> Isn't that technically already possible by doing enough cast and pointer arithmetic hacks? I don't think it really makes a difference. If someone really wants to backdoor a type, they will find a way to do it.

Yes, but that's neither "safe" nor an actual language feature. Allowing RTTI to do the same would be both: my point was whether there should be a language feature for this (exploring unexposed types).

-- 
Best regards,
 Vladimir                            mailto:vladimir@thecybershadow.net

On Sun, 23 Oct 2011 13:48:03 -0400, Jacob Carlborg <doob@me.com> wrote:
> On 2011-10-23 18:03, Robert Jacques wrote:
>> On Sun, 23 Oct 2011 07:06:42 -0400, Timon Gehr <timon.gehr@gmx.ch> wrote:
>> [snip]
>>> The module can generate RTTI for all types recursively from the starting
>>> point iff that information is statically available. It does not have to
>>> be. A module that comes as .di + static library binary could return a
>>> reference to a private class that has a publicly exported base class.
>>> How would you generate RTTI for a statically invisible class?
>>
>> You're not supposed to be able to. Runtime reflection should only apply
>> to public data members.
>
> It's not enough. Take this for example:
>
> class Foo < ActiveRecord::Base
>      before_save :bar
>
>      private
>
>      def bar
>      end
> end
>
> The above code is an example from Ruby on Rails. The "before_save" call
> sets up a callback, "bar", that will called before saving the model. The
> callback method will be called using reflection.

Using a short, cryptic stub from another programming language is not an effective communication tool. Case in point: all I understand is that something is returning a delegate to something, and reflection never enters the picture.

But, for sake of argument, let me assume that ActiveRecord needs to reflect Foo for rails to work (i.e. ActiveRecord::Base's ctor reflects this). At a very fundamental level, this isn't an argument for or against RTTI in D; in D you'd do this with a mixin of some sort. When looking a use cases/features from other languages, always remember to first ask oneself a) how would I do it in D today? b) Is language X's solution 'better, faster, stronger' in some way?

I mean, considering that Ruby is a dynamic language, would bar even be considered private from inside the super-class's ctor? Is Ruby's private even comparable to D's private? Given that Ruby, by design, requires fields to be private and for all external accesses to happen via methods, should we consider a way to violate that contract an example of mis-feature?

On 10/21/2011 12:07 PM, Vladimir Panteleev wrote:
> Walter: would it be okay if the compiler changes were published as a GitHub
> fork, or should we stick to patches?

I'd prefer as a patch, because forking the language can result in another Phobos v Tango debacle, where everyone loses.

On 2011-10-23 20:54, Robert Jacques wrote:
> On Sun, 23 Oct 2011 13:48:03 -0400, Jacob Carlborg <doob@me.com> wrote:
>> On 2011-10-23 18:03, Robert Jacques wrote:
>>> On Sun, 23 Oct 2011 07:06:42 -0400, Timon Gehr <timon.gehr@gmx.ch>
>>> wrote:
>>> [snip]
>>>> The module can generate RTTI for all types recursively from the
>>>> starting
>>>> point iff that information is statically available. It does not have to
>>>> be. A module that comes as .di + static library binary could return a
>>>> reference to a private class that has a publicly exported base class.
>>>> How would you generate RTTI for a statically invisible class?
>>>
>>> You're not supposed to be able to. Runtime reflection should only apply
>>> to public data members.
>>
>> It's not enough. Take this for example:
>>
>> class Foo < ActiveRecord::Base
>> before_save :bar
>>
>> private
>>
>> def bar
>> end
>> end
>>
>> The above code is an example from Ruby on Rails. The "before_save" call
>> sets up a callback, "bar", that will called before saving the model. The
>> callback method will be called using reflection.
>
> Using a short, cryptic stub from another programming language is not an
> effective communication tool. Case in point: all I understand is that
> something is returning a delegate to something, and reflection never
> enters the picture.

What's happening is that the class method "before_save" is called with the symbol (think string) "bar". Then later ActiveRecord::Base will call the method corresponding to the string "bar" using reflection.

In D you would proabably do something like:

class Foo : ActiveRecord.Base
{
    private void bar () {}

    mixin beforeSave!(bar);
}

Now you still won't be able to call "bar" outside "Foo" if "bar" is private.

> But, for sake of argument, let me assume that ActiveRecord needs to
> reflect Foo for rails to work (i.e. ActiveRecord::Base's ctor reflects
> this). At a very fundamental level, this isn't an argument for or
> against RTTI in D; in D you'd do this with a mixin of some sort. When
> looking a use cases/features from other languages, always remember to
> first ask oneself a) how would I do it in D today? b) Is language X's
> solution 'better, faster, stronger' in some way?

You can think of it like this, just because you haven't come up with a reason for calling privte methods using reflection doesn't mean anyone else won't come up with a good reason. I tried here to give a reason.

> I mean, considering that Ruby is a dynamic language, would bar even be
> considered private from inside the super-class's ctor? Is Ruby's private
> even comparable to D's private? Given that Ruby, by design, requires
> fields to be private and for all external accesses to happen via
> methods, should we consider a way to violate that contract an example of
> mis-feature?

There is no constructor involved here. "before_save" is class method called when the class "Foo" is loaded.

You can do basically whatever you want with the code in Ruby. You can replace an existing method/class in the standard library with your own. You can easily add methods to existing classes. It's the programmer that is responsible to not mess up things, just as in D where you can cast away immutable, shared and do unsafe things with pointers.

-- 
/Jacob Carlborg

On 2011-10-23 22:14, Walter Bright wrote:
> On 10/21/2011 12:07 PM, Vladimir Panteleev wrote:
>> Walter: would it be okay if the compiler changes were published as a
>> GitHub
>> fork, or should we stick to patches?
>
> I'd prefer as a patch, because forking the language can result in
> another Phobos v Tango debacle, where everyone loses.

I though we used forks on GitHub with pull requests these days.

-- 
/Jacob Carlborg

On Fri, 21 Oct 2011 23:48:51 -0400, Robert Jacques <sandford@jhu.edu> wrote:

> On Fri, 21 Oct 2011 17:23:17 -0400, Daniel Gibson <metalcaedes@gmail.com> wrote:
>> Am 21.10.2011 21:07, schrieb Vladimir Panteleev:
>>> Hi,
>>>
>>> Igor Stepanov has created a patch for DMD and Druntime which adds RTTI
>>> information for class and struct members.
>>>
>>> Example:
>>>
>>> import std.stdio;
>>>
>>> class Foo
>>> {
>>> static void PrintHello()
>>> {
>>> writeln("Hello");
>>> }
>>> }
>>> void main()
>>> {
>>> auto info = cast(OffsetTypeInfo_StaticMethod)Foo.classinfo.m_offTi[0];
>>> assert(info.name == "PrintHello");
>>> auto print = cast(void function())info.pointer;
>>> print(); //prints "Hello"
>>> }
>>>
>>> While the inclusion of such functionality into the language remains a
>>> disputed matter, would anyone be interested in an unofficial patch for
>>> this?
>>>
>>> Walter: would it be okay if the compiler changes were published as a
>>> GitHub fork, or should we stick to patches?
>>>
>>
>> I'd love to see proper runtime reflection support in D, including
>> functionality to get information about available methods (their name and
>> parameters) and a way to call them.
>
> What do you mean by their 'parameters'? What about overloads? Attributes? Arguments? Argument attributes?
>
>> Something that is close to what Java offers would be great.
>
> And what, exactly does JAVA offer? What works? What doesn't work? What's missing?
>
>> BTW: I don't really see the problem with providing this information
>> (overhead-wise) - the information needs to be available once per
>> class/struct, but objects of classes just need one pointer to it (other
>> types don't even need that because they're not polymorphic and - like
>> methods of structs - the address of the information is known at
>> compile-time).
>
> 1) Unused information is simply bloat: it increases exe size, slows the exe down and increases the runtime memory footprint.
> 2) On a lot of systems (i.e. consoles, embedded, smart phones, tablets) memory and disk space are both highly constrained resources that you don't want to waste.
> 3) RTTI provides a back-door into a code-base; one that for many reasons you may want to keep closed.

I agree to all these points.  I think runtime reflection should be generated by the compiler as needed.  And the mechanism to determine "as needed" should be an attribute IMO, or an intrinsic function (i.e. if you have all the static information about a type, you should be able to generate the RTTI for it).  I do not want programs getting bigger than they already are for the sake of "just in case I want to use reflection"...

-Steve

Le 24/10/2011 16:20, Steven Schveighoffer a écrit :
> On Fri, 21 Oct 2011 23:48:51 -0400, Robert Jacques <sandford@jhu.edu>
> wrote:
>
>> On Fri, 21 Oct 2011 17:23:17 -0400, Daniel Gibson
>> <metalcaedes@gmail.com> wrote:
>>> Am 21.10.2011 21:07, schrieb Vladimir Panteleev:
>>>> Hi,
>>>>
>>>> Igor Stepanov has created a patch for DMD and Druntime which adds RTTI
>>>> information for class and struct members.
>>>>
>>>> Example:
>>>>
>>>> import std.stdio;
>>>>
>>>> class Foo
>>>> {
>>>> static void PrintHello()
>>>> {
>>>> writeln("Hello");
>>>> }
>>>> }
>>>> void main()
>>>> {
>>>> auto info = cast(OffsetTypeInfo_StaticMethod)Foo.classinfo.m_offTi[0];
>>>> assert(info.name == "PrintHello");
>>>> auto print = cast(void function())info.pointer;
>>>> print(); //prints "Hello"
>>>> }
>>>>
>>>> While the inclusion of such functionality into the language remains a
>>>> disputed matter, would anyone be interested in an unofficial patch for
>>>> this?
>>>>
>>>> Walter: would it be okay if the compiler changes were published as a
>>>> GitHub fork, or should we stick to patches?
>>>>
>>>
>>> I'd love to see proper runtime reflection support in D, including
>>> functionality to get information about available methods (their name and
>>> parameters) and a way to call them.
>>
>> What do you mean by their 'parameters'? What about overloads?
>> Attributes? Arguments? Argument attributes?
>>
>>> Something that is close to what Java offers would be great.
>>
>> And what, exactly does JAVA offer? What works? What doesn't work?
>> What's missing?
>>
>>> BTW: I don't really see the problem with providing this information
>>> (overhead-wise) - the information needs to be available once per
>>> class/struct, but objects of classes just need one pointer to it (other
>>> types don't even need that because they're not polymorphic and - like
>>> methods of structs - the address of the information is known at
>>> compile-time).
>>
>> 1) Unused information is simply bloat: it increases exe size, slows
>> the exe down and increases the runtime memory footprint.
>> 2) On a lot of systems (i.e. consoles, embedded, smart phones,
>> tablets) memory and disk space are both highly constrained resources
>> that you don't want to waste.
>> 3) RTTI provides a back-door into a code-base; one that for many
>> reasons you may want to keep closed.
>
> I agree to all these points. I think runtime reflection should be
> generated by the compiler as needed. And the mechanism to determine "as
> needed" should be an attribute IMO, or an intrinsic function (i.e. if
> you have all the static information about a type, you should be able to
> generate the RTTI for it). I do not want programs getting bigger than
> they already are for the sake of "just in case I want to use reflection"...
>
> -Steve

Well, if the compile time reflection is good enough, you can do this without any runtime reflexion support in the compiler, just using a lib.

Why not support this option ?

On Mon, 24 Oct 2011 03:30:02 -0400, Jacob Carlborg <doob@me.com> wrote:
> On 2011-10-23 20:54, Robert Jacques wrote:
>> On Sun, 23 Oct 2011 13:48:03 -0400, Jacob Carlborg <doob@me.com> wrote:
>>> On 2011-10-23 18:03, Robert Jacques wrote:
>>>> On Sun, 23 Oct 2011 07:06:42 -0400, Timon Gehr <timon.gehr@gmx.ch>
>>>> wrote:
>>>> [snip]
>>>>> The module can generate RTTI for all types recursively from the
>>>>> starting
>>>>> point iff that information is statically available. It does not have to
>>>>> be. A module that comes as .di + static library binary could return a
>>>>> reference to a private class that has a publicly exported base class.
>>>>> How would you generate RTTI for a statically invisible class?
>>>>
>>>> You're not supposed to be able to. Runtime reflection should only apply
>>>> to public data members.
>>>
>>> It's not enough. Take this for example:
>>>
>>> class Foo < ActiveRecord::Base
>>> before_save :bar
>>>
>>> private
>>>
>>> def bar
>>> end
>>> end
>>>
>>> The above code is an example from Ruby on Rails. The "before_save" call
>>> sets up a callback, "bar", that will called before saving the model. The
>>> callback method will be called using reflection.
>>
>> Using a short, cryptic stub from another programming language is not an
>> effective communication tool. Case in point: all I understand is that
>> something is returning a delegate to something, and reflection never
>> enters the picture.
>
> What's happening is that the class method "before_save" is called with
> the symbol (think string) "bar". Then later ActiveRecord::Base will call
> the method corresponding to the string "bar" using reflection.
>
> In D you would proabably do something like:
>
> class Foo : ActiveRecord.Base
> {
>      private void bar () {}
>
>      mixin beforeSave!(bar);
> }
>
> Now you still won't be able to call "bar" outside "Foo" if "bar" is private.

Thanks. But how does super know to call "bar"? And if you can pass information to super, couldn't you just pass a delegate? And isn't passing information to super simply passing information to yourself?

>> But, for sake of argument, let me assume that ActiveRecord needs to
>> reflect Foo for rails to work (i.e. ActiveRecord::Base's ctor reflects
>> this). At a very fundamental level, this isn't an argument for or
>> against RTTI in D; in D you'd do this with a mixin of some sort. When
>> looking a use cases/features from other languages, always remember to
>> first ask oneself a) how would I do it in D today? b) Is language X's
>> solution 'better, faster, stronger' in some way?
>
> You can think of it like this, just because you haven't come up with a
> reason for calling privte methods using reflection doesn't mean anyone
> else won't come up with a good reason. I tried here to give a reason.

I'm sorry if I've come across that way. I'm well aware with the reasons for wanting access to private methods/fields via reflection and have mentioned them in previous posts. What I've tried to point out is very eloquently stated by fig. 6.3 on page 204 of TDPL: allowing reflection to bypass protection guarantees essentially makes all declarations public in nature, if not extern. That's a very heavy price to pay, just from a program maintenance perspective. And if you consider someone writing medical or financial software, the privacy concerns of exposing private variable to all become very real.

Back to my actual argument, what I was pointing out was that the toolboxes of various languages differ heavily. Just because Ruby has a very cool library which uses runtime reflection, one shouldn't simply state 'D must have runtime reflection.' One needs to first decompose what the library is trying to achieve from the language mechanisms used to implement it. Then you can ask yourself 1) Can I do that in D? 2) How does D's solution compare to Ruby's?

My impression of what ActiveRecord, (which my be wrong), is that its using runtime-reflection to do various meta-programming activities and that under-the-hood, it stretching Ruby to the breaking point in order to do so. D, on the other hand, has a ton of meta-programming specific features, and doesn't need to rely on reflection to do so. This is partly to do with Ruby supporting 'eval' and thus making no distinction between compile-time and run-time behavior; that's correct for Ruby, but D is a different kettle of fish.

>> I mean, considering that Ruby is a dynamic language, would bar even be
>> considered private from inside the super-class's ctor? Is Ruby's private
>> even comparable to D's private? Given that Ruby, by design, requires
>> fields to be private and for all external accesses to happen via
>> methods, should we consider a way to violate that contract an example of
>> mis-feature?
>
> There is no constructor involved here. "before_save" is class method
> called when the class "Foo" is loaded.

So it's the static ctor, then.

> You can do basically whatever you want with the code in Ruby. You can
> replace an existing method/class in the standard library with your own.
> You can easily add methods to existing classes. It's the programmer that
> is responsible to not mess up things, just as in D where you can cast
> away immutable, shared and do unsafe things with pointers.

Well one one hand, none of those things are allowed in SafeD (And you know, that's not a resounding argument for reflection of private members). On the other, you didn't answered my other two questions. In fact, the impression I'm getting is that Ruby's private isn't comparable to D's private.

On Mon, 24 Oct 2011 11:18:34 -0400, deadalnix <deadalnix@gmail.com> wrote:
[snip]

> Well, if the compile time reflection is good enough, you can do this
> without any runtime reflexion support in the compiler, just using a lib.
>
> Why not support this option ?

Some of us are: https://jshare.johnshopkins.edu/rjacque2/public_html/variant.mht

On Mon, 24 Oct 2011 11:18:34 -0400, deadalnix <deadalnix@gmail.com> wrote:

> Le 24/10/2011 16:20, Steven Schveighoffer a écrit :
>> On Fri, 21 Oct 2011 23:48:51 -0400, Robert Jacques <sandford@jhu.edu>
>> wrote:
>>
>>> On Fri, 21 Oct 2011 17:23:17 -0400, Daniel Gibson
>>> <metalcaedes@gmail.com> wrote:
>>>> Am 21.10.2011 21:07, schrieb Vladimir Panteleev:
>>>>> Hi,
>>>>>
>>>>> Igor Stepanov has created a patch for DMD and Druntime which adds RTTI
>>>>> information for class and struct members.
>>>>>
>>>>> Example:
>>>>>
>>>>> import std.stdio;
>>>>>
>>>>> class Foo
>>>>> {
>>>>> static void PrintHello()
>>>>> {
>>>>> writeln("Hello");
>>>>> }
>>>>> }
>>>>> void main()
>>>>> {
>>>>> auto info = cast(OffsetTypeInfo_StaticMethod)Foo.classinfo.m_offTi[0];
>>>>> assert(info.name == "PrintHello");
>>>>> auto print = cast(void function())info.pointer;
>>>>> print(); //prints "Hello"
>>>>> }
>>>>>
>>>>> While the inclusion of such functionality into the language remains a
>>>>> disputed matter, would anyone be interested in an unofficial patch for
>>>>> this?
>>>>>
>>>>> Walter: would it be okay if the compiler changes were published as a
>>>>> GitHub fork, or should we stick to patches?
>>>>>
>>>>
>>>> I'd love to see proper runtime reflection support in D, including
>>>> functionality to get information about available methods (their name and
>>>> parameters) and a way to call them.
>>>
>>> What do you mean by their 'parameters'? What about overloads?
>>> Attributes? Arguments? Argument attributes?
>>>
>>>> Something that is close to what Java offers would be great.
>>>
>>> And what, exactly does JAVA offer? What works? What doesn't work?
>>> What's missing?
>>>
>>>> BTW: I don't really see the problem with providing this information
>>>> (overhead-wise) - the information needs to be available once per
>>>> class/struct, but objects of classes just need one pointer to it (other
>>>> types don't even need that because they're not polymorphic and - like
>>>> methods of structs - the address of the information is known at
>>>> compile-time).
>>>
>>> 1) Unused information is simply bloat: it increases exe size, slows
>>> the exe down and increases the runtime memory footprint.
>>> 2) On a lot of systems (i.e. consoles, embedded, smart phones,
>>> tablets) memory and disk space are both highly constrained resources
>>> that you don't want to waste.
>>> 3) RTTI provides a back-door into a code-base; one that for many
>>> reasons you may want to keep closed.
>>
>> I agree to all these points. I think runtime reflection should be
>> generated by the compiler as needed. And the mechanism to determine "as
>> needed" should be an attribute IMO, or an intrinsic function (i.e. if
>> you have all the static information about a type, you should be able to
>> generate the RTTI for it). I do not want programs getting bigger than
>> they already are for the sake of "just in case I want to use reflection"...
>>
>> -Steve
>
> Well, if the compile time reflection is good enough, you can do this without any runtime reflexion support in the compiler, just using a lib.
>
> Why not support this option ?

As others have mentioned, there should be a standard way so we don't have conflicting ways that don't mix.

Plus, it seems logical tome since the compiler is generating *some* RTTI, it should generate *all* of it (when asked to do so).

-Steve