Are there any clever ways of preserving purity when memoizing functions in D?
I want this when caching SHA1-calculations of large datasets kept in RAM.
Short answer: Pick memoization or purity. Don't try and have both.
Long answer: I don't see how it would be possible to preserve purity with memoization unless you used casts to lie to the compiler and claim that a function is pure when it isn't, because in order to memoize, you have to store the arguments and the result, which breaks purity, since the number one guarantee of pure functions is that they don't access mutable global or static variables (which is the only way that you'd be able to memoize anything).
So, if you did something like
alias pure nothrow Foo function() FuncType;
auto result = (cast(FuncType)&theFunc)();
then you can treat theFunc as if it were pure when it isn't, but then it's up to you to ensure that the function acts pure from the outside - including dealing with the fact that the compiler thinks that it can change the mutability of the return type of a strongly pure function which returns a mutable type. For instance, this code will compile just fine
char[] makeString(size_t len) pure
{
return new char[](len);
}
void main()
{
char[] a = makeString(5);
const(char)[] b = makeString(5);
const(char[]) c = makeString(5);
immutable(char)[] d = makeString(5);
immutable(char[]) e = makeString(5);
}
even though the return type is always mutable. And that's because the compiler knows that makeString is strongly pure and returns a value which could not have been passed to it - so, it's guaranteed to be a new value every time - and therefore changing changing the mutability of the return type to const or immutable doesn't violate the type system.
If you were to do something inside of makeString that involved casting a function to pure when it violated the guarantee that makeString always returned a new value, then you'd have broken the type system, and you'd be risking having very buggy code depending on what you did with the values returned from makeString.
The only way that I'm aware of getting purity when you don't have it is to cast a function pointer so that it's pure, but if you do that, then you must fully understand what guarantees a pure function makes and what the compiler thinks that it can do with it so that you fully mimic that behavior. That's easier if you're returning immutable data or a value type, because then you don't have the issue of the compiler changing the mutability of the return type, but it's still very tricky business.
So, if you're thinking about casting something to pure, think again. Yes, it's possible to do some stuff that way that you couldn't otherwise, but it's very risky. Personally, I'd advise that you decide whether purity matters more to you or memoization matters more to you and that you drop the other. Anything else is highly risky.
What D allows to express within the type system is an impure function that memoizes a pure one.
Conceptually the memoizer is also pure, but the type system is not sufficiently expressive to allow that. You'd need to cheat somewhere.
Related
More than often I find myself passing a variable into a method, where it is not directly used, but is yet passed to another method for further processing. I don't know if there is a name for this practice either. I can explain it better with a sample code:
static void main() {
int a = method1(var1, var2, var3)
}
int method1(int var1, int var2, int var3) {
var4 = some_function_of(var1, var2)
return method2(var3, var4)
}
int method2(int var 3, int var4) {
return some_other_function_of(var3, var4)
}
This case can be expanded where there the same variable (var3) is passed through even longer chains of methods. I think this could be a bad practice, as in my example method1 is some sort of intermediate that is not manipulating var3. Is there a better practice for performance, design and/or readability?
At least for object oriented languages the answer would be:
You definitely want to avoid such code - as you struggle to reduce your parameter list to the absolut minimum; the goal is zero.
If you find that your class should offer various methods that all require the "same" parameter; than that makes that parameter a candidate to be a field within your class.
In non-oo languages, I think you have to pragmatically balance between having more functions and parameter list length. In your example,
static void main() {
int var4 = some_function_of(var1, var2)
int a = method2(var3, var4)
}
avoiding method1 ... saves you passing var3 to your first method. And you are still within the rules of the "single layer of abstraction" principle.
This is not at all uncommon and not necessarily a bad practice. It can impact all three of the metrics you mentioned though:
Performance: Adding another parameter to a function call may result in a performance hit but not always. It depends on the language, compiler/interpreter, and platform. For example, an optimizing compiler for C++ will try to avoid copying a variable even if it is passed by value if it can (sometimes it will eliminate a function call completely). But passing a value through multiple functions might mess up the compiler's optimizations if it can't follow the path well. Still, I expect any performance hit from this to be minimal.
Design: Depending on your language's paradigm (object oriented, functional, etc...) this might indicate that your design could be improved, perhaps by encapsulating the data in a structure or class so that only one parameter is passed (a class instance pointer) and each function accesses only the class members it needs.
Readability: This shouldn't make the individual functions harder to read, since they shouldn't care where parameters come from and it is clear that the parameter is being passed to another function. It could make it harder to understand the whole program though because it can be hard to keep track of where values originate if they are passed through a long chain of calls before being touched.
In general, it is good to minimize the parameter list (for all of these reasons) and to keep data "closer" to code that needs it. If you do those things, this case shouldn't pop up much and when it does it will be less likely to be due to bad design.
Initial note: I'm working in Julia, but this question probably applies to many languages.
Setup: I have a composite type as follows:
type MyType
x::Vector{String}
end
I write some methods to act on MyType. For example, I write a method that allows me to insert a new element in x, e.g. function insert!(d::MyType, itemToInsert::String).
Question: Should MyType be mutable or immutable?
My understanding: I've read the Julia docs on this, as well as more general (and highly upvoted) questions on Stackoverflow (e.g. here or here), but I still don't really have a good handle on what it means to be mutable/immutable from a practical perspective (especially for the case of an immutable composite type, containing a mutable array of immutable types!)
Nonetheless, here is my attempt: If MyType is immutable, then it means that the field x must always point to the same object. That object itself (a vector of Strings) is mutable, so it is perfectly okay for me to insert new elements into it. What I am not allowed to do is try and alter MyType so that the field x points to an entirely different object. For example, methods that do the following are okay:
MyType.x[1] = "NewValue"
push!(MyType.x, "NewElementToAdd")
But methods that do the following are not okay:
MyType.x = ["a", "different", "string", "array"]
Is this right? Also, is the idea that the object that an immutable types field values are locked to are those that are created within the constructor?
Final Point: I apologise if this appears to duplicate other questions on SO. As stated, I have looked through them and wasn't able to get the understanding that I was after.
So here is something mind bending to consider (at least to me):
julia> immutable Foo
data::Vector{Float64}
end
julia> x = Foo([1.0, 2.0, 4.0])
Foo([1.0,2.0,4.0])
julia> append!(x.data, x.data); pointer(x.data)
Ptr{Float64} #0x00007ffbc3332018
julia> append!(x.data, x.data); pointer(x.data)
Ptr{Float64} #0x00007ffbc296ac28
julia> append!(x.data, x.data); pointer(x.data)
Ptr{Float64} #0x00007ffbc34809d8
So the data address is actually changing as the vector grows and needs to be reallocated! But - you can't change data yourself, as you point out.
I'm not sure there is a 100% right answer is really. I primarily use immutable for simple types like the Complex example in the docs in some performance critical situations, and I do it for "defensive programming" reasons, e.g. the code has no need to write to the fields of this type so I make it an error to do so. They are a good choice IMO whenever the type is a sort of an extension of a number, e.g. Complex, RGBColor, and I use them in place of tuples, as a kind of named tuple (tuples don't seem to perform well with Julia right now anyway, wheres immutable types perform excellently).
so i just came across some code that reads like so:
checkCalculationPeriodFrequency("7D", "7D", SHOULD_MATCH);
and
checkCalculationPeriodFrequency("7D", "8D", SHOULD_NOT_MATCH);
Let's not worry about what the code does for now (or indeed, ever), but instead, let's worry about that last parameter - the SHOULD_MATCH and SHOULD_NOT_MATCH
Its something i've thought of before but thought might be "bad" to do (inasmuch as "bad" holds any real meaning in a postmodernist world).
above, those values are declared (as you might have assumed):
private boolean SHOULD_MATCH = true;
private boolean SHOULD_NOT_MATCH = false;
I can't recall reading about "naming" the boolean parameter passed to a method call to ease readability, but it certainly makes sense (for readability, but then, it also hides what the value is, if only a teeny bit). Is this a style thing that others have found is instagram or like, soooo facebook?
Naming the argument would help with readability, especially when the alternative is usually something like
checkCalculationFrequency("7D",
"8D",
true /* should match */);
which is ugly. Having context-specific constants could be a solution to this.
I would actually go a step further and redefine the function prototype to accept an enum instead:
enum MatchType {
ShouldMatch,
ShouldNotMatch
};
void checkCalculationFrequency(string a, string b, MatchType match);
I would prefer this over a boolean, because it gives you flexibility to extend the function to accept other MatchTypes later.
I suggest you not to do this way.
First, for each object, the two members SHOULD_MATCH and SHOULD_NOT_MATCH are regenerated. And that's not good because it's not a behavior of the object. So it you want to use is, at least describe it as STATIC FINAL.
Second, I prefer to use an enum instead, because you can control completely the value of the param, i.e. when you use it, you must use either SHOULD_MATCH or SHOULD_NOT_MATCH, not just true or false. And this increase the readability too.
Regards.
It is indeed for readability. The idea is that the reader of the function call might not know immediately what the value true mean in the function call, but SHOULD_MATCH conveys the meaning immediately (and if you need to look up the actual value, you can do so with not much effort).
This becomes even more understandable if you have more than one boolean parameters in the function call: which true means what?
The next step in this logic is to create named object values (e.g. via enum) for the parameter values: you cannot pass on the wrong value to the function (e.g. in the example of three boolean parameters, nothing stops me from passing in SHOULD_MATCH for all of them, even though it does not make sense semantically for that function).
It's definitely more than a style thing.
We have a similar system that takes takes input from a switch in the form of boolean values, 1 or 0, which is pretty much the same as true or false.
In this system we declare our variables OPEN = true and CLOSED = false* and pass them into functions which perform different actions depending on the state of the switch. Now if someone happens to hook up the switch differently it may be that we now get the value 0 when it is OPEN and 1 when it is CLOSED.
By having named boolean variables we can easily adapt the system without having to change the logic throughout. The code becomes self documenting because developers can clearer see what action is meant to be taken in which case without worrying what value comes.
Of course the true purpose of the boolean value should be well documented else where and it is in our system....honest....
*(maybe we use OPEN, !OPEN I forget)
Something like this (yes, this doesn't deal with some edge cases - that's not the point):
int CountDigits(int num) {
int count = 1;
while (num >= 10) {
count++;
num /= 10;
}
return count;
}
What's your opinion about this? That is, using function arguments as local variables.
Both are placed on the stack, and pretty much identical performance wise, I'm wondering about the best-practices aspects of this.
I feel like an idiot when I add an additional and quite redundant line to that function consisting of int numCopy = num, however it does bug me.
What do you think? Should this be avoided?
As a general rule, I wouldn't use a function parameter as a local processing variable, i.e. I treat function parameters as read-only.
In my mind, intuitively understandabie code is paramount for maintainability, and modifying a function parameter to use as a local processing variable tends to run counter to that goal. I have come to expect that a parameter will have the same value in the middle and bottom of a method as it does at the top. Plus, an aptly-named local processing variable may improve understandability.
Still, as #Stewart says, this rule is more or less important depending on the length and complexity of the function. For short simple functions like the one you show, simply using the parameter itself may be easier to understand than introducing a new local variable (very subjective).
Nevertheless, if I were to write something as simple as countDigits(), I'd tend to use a remainingBalance local processing variable in lieu of modifying the num parameter as part of local processing - just seems clearer to me.
Sometimes, I will modify a local parameter at the beginning of a method to normalize the parameter:
void saveName(String name) {
name = (name != null ? name.trim() : "");
...
}
I rationalize that this is okay because:
a. it is easy to see at the top of the method,
b. the parameter maintains its the original conceptual intent, and
c. the parameter is stable for the rest of the method
Then again, half the time, I'm just as apt to use a local variable anyway, just to get a couple of extra finals in there (okay, that's a bad reason, but I like final):
void saveName(final String name) {
final String normalizedName = (name != null ? name.trim() : "");
...
}
If, 99% of the time, the code leaves function parameters unmodified (i.e. mutating parameters are unintuitive or unexpected for this code base) , then, during that other 1% of the time, dropping a quick comment about a mutating parameter at the top of a long/complex function could be a big boon to understandability:
int CountDigits(int num) {
// num is consumed
int count = 1;
while (num >= 10) {
count++;
num /= 10;
}
return count;
}
P.S. :-)
parameters vs arguments
http://en.wikipedia.org/wiki/Parameter_(computer_science)#Parameters_and_arguments
These two terms are sometimes loosely used interchangeably; in particular, "argument" is sometimes used in place of "parameter". Nevertheless, there is a difference. Properly, parameters appear in procedure definitions; arguments appear in procedure calls.
So,
int foo(int bar)
bar is a parameter.
int x = 5
int y = foo(x)
The value of x is the argument for the bar parameter.
It always feels a little funny to me when I do this, but that's not really a good reason to avoid it.
One reason you might potentially want to avoid it is for debugging purposes. Being able to tell the difference between "scratchpad" variables and the input to the function can be very useful when you're halfway through debugging.
I can't say it's something that comes up very often in my experience - and often you can find that it's worth introducing another variable just for the sake of having a different name, but if the code which is otherwise cleanest ends up changing the value of the variable, then so be it.
One situation where this can come up and be entirely reasonable is where you've got some value meaning "use the default" (typically a null reference in a language like Java or C#). In that case I think it's entirely reasonable to modify the value of the parameter to the "real" default value. This is particularly useful in C# 4 where you can have optional parameters, but the default value has to be a constant:
For example:
public static void WriteText(string file, string text, Encoding encoding = null)
{
// Null means "use the default" which we would document to be UTF-8
encoding = encoding ?? Encoding.UTF8;
// Rest of code here
}
About C and C++:
My opinion is that using the parameter as a local variable of the function is fine because it is a local variable already. Why then not use it as such?
I feel silly too when copying the parameter into a new local variable just to have a modifiable variable to work with.
But I think this is pretty much a personal opinion. Do it as you like. If you feel sill copying the parameter just because of this, it indicates your personality doesn't like it and then you shouldn't do it.
If I don't need a copy of the original value, I don't declare a new variable.
IMO I don't think mutating the parameter values is a bad practice in general,
it depends on how you're going to use it in your code.
My team coding standard recommends against this because it can get out of hand. To my mind for a function like the one you show, it doesn't hurt because everyone can see what is going on. The problem is that with time functions get longer, and they get bug fixes in them. As soon as a function is more than one screen full of code, this starts to get confusing which is why our coding standard bans it.
The compiler ought to be able to get rid of the redundant variable quite easily, so it has no efficiency impact. It is probably just between you and your code reviewer whether this is OK or not.
I would generally not change the parameter value within the function. If at some point later in the function you need to refer to the original value, you still have it. in your simple case, there is no problem, but if you add more code later, you may refer to 'num' without realizing it has been changed.
The code needs to be as self sufficient as possible. What I mean by that is you now have a dependency on what is being passed in as part of your algorithm. If another member of your team decides to change this to a pass by reference then you might have big problems.
The best practice is definitely to copy the inbound parameters if you expect them to be immutable.
I typically don't modify function parameters, unless they're pointers, in which case I might alter the value that's pointed to.
I think the best-practices of this varies by language. For example, in Perl you can localize any variable or even part of a variable to a local scope, so that changing it in that scope will not have any affect outside of it:
sub my_function
{
my ($arg1, $arg2) = #_; # get the local variables off the stack
local $arg1; # changing $arg1 here will not be visible outside this scope
$arg1++;
local $arg2->{key1}; # only the key1 portion of the hashref referenced by $arg2 is localized
$arg2->{key1}->{key2} = 'foo'; # this change is not visible outside the function
}
Occasionally I have been bitten by forgetting to localize a data structure that was passed by reference to a function, that I changed inside the function. Conversely, I have also returned a data structure as a function result that was shared among multiple systems and the caller then proceeded to change the data by mistake, affecting these other systems in a difficult-to-trace problem usually called action at a distance. The best thing to do here would be to make a clone of the data before returning it*, or make it read-only**.
* In Perl, see the function dclone() in the built-in Storable module.
** In Perl, see lock_hash() or lock_hash_ref() in the built-in Hash::Util module).
This is general programming, but if it makes a difference, I'm using objective-c. Suppose there's a method that returns a value, and also performs some actions, but you don't care about the value it returns, only the stuff that it does. Would you just call the method as if it was void? Or place the result in a variable and then delete it or forget about it? State your opinion, what you would do if you had this situation.
A common example of this is printf, which returns an int... but you rarely see this:
int val = printf("Hello World");
Yeah just call the method as if it was void. You probably do it all the time without noticing it. The assignment operator '=' actually returns a value, but it's very rarely used.
It depends on the environment (the language, the tools, the coding standard, ...).
For example in C, it is perfectly possible to call a function without using its value. With some functions like printf, which returns an int, it is done all the time.
Sometimes not using a value will cause a warning, which is undesirable. Assigning the value to a variable and then not using it will just cause another warning about an unused variable. For this case the solution is to cast the result to void by prefixing the call with (void), e.g.
(void) my_function_returning_a_value_i_want_to_ignore().
There are two separate issues here, actually:
Should you care about returned value?
Should you assign it to a variable you're not going to use?
The answer to #2 is a resounding "NO" - unless, of course, you're working with a language where that would be illegal (early Turbo Pascal comes to mind). There's absolutely no point in defining a variable only to throw it away.
First part is not so easy. Generally, there is a reason value is returned - for idempotent functions the result is function's sole purpose; for non-idempotent it usually represents some sort of return code signifying whether operation was completed normally. There are exceptions, of course - like method chaining.
If this is common in .Net (for example), there's probably an issue with the code breaking CQS.
When I call a function that returns a value that I ignore, it's usually because I'm doing it in a test to verify behavior. Here's an example in C#:
[Fact]
public void StatService_should_call_StatValueRepository_for_GetPercentageValues()
{
var statValueRepository = new Mock<IStatValueRepository>();
new StatService(null, statValueRepository.Object).GetValuesOf<PercentageStatValue>();
statValueRepository.Verify(x => x.GetStatValues());
}
I don't really care about the return type, I just want to verify that a method was called on a fake object.
In C it is very common, but there are places where it is ok to do so and other places where it really isn't. Later versions of GCC have a function attribute so that you can get a warning when a function is used without checking the return value:
The warn_unused_result attribute causes a warning to be emitted if a caller of the function with this attribute does not use its return value. This is useful for functions where not checking the result is either a security problem or always a bug, such as realloc.
int fn () __attribute__ ((warn_unused_result));
int foo ()
{
if (fn () < 0) return -1;
fn ();
return 0;
}
results in warning on line 5.
Last time I used this there was no way of turning off the generated warning, which causes problems when you're compiling 3rd-party code you don't want to modify. Also, there is of course no way to check if the user actually does something sensible with the returned value.