Related
In imperative programming, there is concise syntax sugar for changing part of an object, e.g. assigning to a field:
foo.bar = new_value
Or to an element of an array, or in some languages an array-like list:
a[3] = new_value
In functional programming, the idiom is not to mutate part of an existing object, but to create a new object with most of the same values, but a different value for that field or element.
At the semantic level, this brings about significant improvements in ease of understanding and composing code, albeit not without trade-offs.
I am asking here about the trade-offs at the syntax level. In general, creating a new object with most of the same values, but a different value for one field or element, is a much more heavyweight operation in terms of how it looks in your code.
Is there any functional programming language that provides syntax sugar to make that operation look more concise? Obviously you can write a function to do it, but imperative languages provide syntax sugar to make it more concise than calling a procedure; do any functional languages provide syntax sugar to make it more concise than calling a function? I could swear that I have seen syntax sugar for at least the object.field case, in some functional language, though I forget which one it was.
(Performance is out of scope here. In this context, I am talking only about what the code looks like and does, not how fast it does it.)
Haskell records have this functionality. You can define a record to be:
data Person = Person
{ name :: String
, age :: Int
}
And an instance:
johnSmith :: Person
johnSmith = Person
{ name = "John Smith"
, age = 24
}
And create an alternation:
johnDoe :: Person
johnDoe = johnSmith {name = "John Doe"}
-- Result:
-- johnDoe = Person
-- { name = "John Doe"
-- , age = 24
-- }
This syntax, however, is cumbersome when you have to update deeply nested records. We've got a library lens that solves this problem quite well.
However, Haskell lists do not provide an update syntax because updating on lists will have an O(n) cost - they are singly-linked lists.
If you want efficient update on list-like collections, you can use Arrays in the array package, or Vectors in the vector package. They both have the infix operator (//) for updating:
alteredVector = someVector // [(1, "some value")]
-- similar to `someVector[1] = "some value"`
it is not built-in, but I think infix notation is convenient enough!
One language with that kind of sugar is F#. It allows you to write
let myRecord3 = { myRecord2 with Y = 100; Z = 2 }
Scala also has sugar for updating a Map:
ms + (k -> v)
ms updated (k,v)
In a language such as Haskell, you would need to write this yourself. If you can express the update as a key-value pair, you might define
let structure' =
update structure key value
or
update structure (key, value)
which would let you use infix notation such as
structure `update` (key, value)
structure // (key, value)
As a proof of concept, here is one possible (inefficient) implementation, which also fails if your index is out of range:
module UpdateList (updateList, (//)) where
import Data.List (splitAt)
updateList :: [a] -> (Int,a) -> [a]
updateList xs (i,y) = let ( initial, (_:final) ) = splitAt i xs
in initial ++ (y:final)
infixl 6 // -- Same precedence as +
(//) :: [a] -> (Int,a) -> [a]
(//) = updateList
With this definition, ["a","b","c","d"] // (2,"C") returns ["a","b","C","d"]. And [1,2] // (2,3) throws a runtime exception, but I leave that as an exercise for the reader.
H. Rhen gave an example of Haskell record syntax that I did not know about, so I’ve removed the last part of my answer. See theirs instead.
I'm trying to build a simple Bayesian network, where rain and sprinkler are the parents of wetgrass, but rain and sprinkler each have three (fuzzy-logic type rather rather than the usual two boolean) states, and wetgrass has two states (true/false). I can't find anywhere in the pymc3 docs what syntax to use to describe the CPTs for this -- I'm trying the following based on 2-state examples but it's not generalizing to three states the way I thought it would. Can anyone show the correct way to do this? (And also for the more general case where wetgrass has three states too.)
rain = mc.Categorical('rain', p = np.array([0.5, 0. ,0.5]))
sprinker = mc.Categorical('sprinkler', p=np.array([0.33,0.33,0.34]))
wetgrass = mc.Categorical('wetgrass',
mc.math.switch(rain,
mc.math.switch(sprinker, 10, 1, -4),
mc.math.switch(sprinker, -20, 1, 3),
mc.math.switch(sprinker, -5, 1, -0.5)))
[gives error at wetgrass definition:
Wrong number of inputs for Switch.make_node (got 4((, , , )), expected 3)
]
As I understand it - switch is a theano function similar to (b?a:b) in a C program; which is only doing a two way comparison. It's maybe possible to set up the CPT using a whole load of binary switches like this, but I really want to just give a 3D matrix CPT as the input as in BNT and other bayes net libraries. Is this currently possible ?
You can code a three-way switch using two individual switches:
tt.switch(sprinker == 0,
10
tt.switch(sprinker == 1, 1, -4))
But in general it is probably better to index into a table:
table = tt.constant(np.array([[...], [...]]))
value = table[rain, sprinker]
I am looking for a general-purpose way of defining textual expressions which allow a value to be validated.
For example, I have a value which should only be set to 1, 2, 3, 10, 11, or 12.
Its constraint might be defined as: (value >= 1 && value <= 3) || (value >= 10 && value <= 12)
Or another value which can be 1, 3, 5, 7, 9 etc... would have a constraint like value % 2 == 1 or IsOdd(value).
(To help the user correct invalid values, I'd like to show the constraint - so something descriptive like IsOdd is preferable.)
These constraints would be evaluated both on client-side (after user input) and server-side.
Therefore a multi-platform solution would be ideal (specifically Win C#/Linux C++).
Is there an existing language/project which allows evaluation or parsing of similar simple expressions?
If not, where might I start creating my own?
I realise this question is somewhat vague as I am not entirely sure what I am after. Searching turned up no results, so even some terms as a starting point would be helpful. I can then update/tag the question accordingly.
You may want to investigate dependently typed languages like Idris or Agda.
The type system of such languages allows encoding of value constraints in types. Programs that cannot guarantee the constraints will simply not compile. The usual example is that of matrix multiplication, where the dimensions must match. But this is so to speak the "hello world" of dependently typed languages, the type system can do much more for you.
If you end up starting your own language I'd try to stay implementation-independent as long as possible. Look for the formal expression grammars of a suitable programming language (e.g. C) and add special keywords/functions as required. Once you have a formal definition of your language, implement a parser using your favourite parser generator.
That way, even if your parser is not portable to a certain platform you at least have a formal standard from where to start a separate parser implementation.
You may also want to look at creating a Domain Specific Language (DSL) in Ruby. (Here's a good article on what that means and what it would look like: http://jroller.com/rolsen/entry/building_a_dsl_in_ruby)
This would definitely give you the portability you're looking for, including maybe using IronRuby in your C# environment, and you'd be able to leverage the existing logic and mathematical operations of Ruby. You could then have constraint definition files that looked like this:
constrain 'wakeup_time' do
6 <= value && value <= 10
end
constrain 'something_else' do
check (value % 2 == 1), MustBeOdd
end
# constrain is a method that takes one argument and a code block
# check is a function you've defined that takes a two arguments
# MustBeOdd is the name of an exception type you've created in your standard set
But really, the great thing about a DSL is that you have a lot of control over what the constraint files look like.
there are a number of ways to verify a list of values across multiple languages. My preferred method is to make a list of the permitted values and load them into a dictionary/hashmap/list/vector (dependant on the language and your preference) and write a simple isIn() or isValid() function, that will check that the value supplied is valid based on its presence in the data structure. The beauty of this is that the code is trivial and can be implemented in just about any language very easily. for odd-only or even-only numeric validity again, a small library of different language isOdd() functions will suffice: if it isn't odd it must by definition be even (apart from 0 but then a simple exception can be set up to handle that, or you can simply specify in your code documentation that for logical purposes your code evaluates 0 as odd/even (your choice)).
I normally cart around a set of c++ and c# functions to evaluate isOdd() for similar reasons to what you have alluded to, and the code is as follows:
C++
bool isOdd( int integer ){ return (integer%2==0)?false:true; }
you can also add inline and/or fastcall to the function depending on need or preference; I tend to use it as an inline and fastcall unless there is a need to do otherwise (huge performance boost on xeon processors).
C#
Beautifully the same line works in C# just add static to the front if it is not going to be part of another class:
static bool isOdd( int integer ){ return (integer%2==0)?false:true; }
Hope this helps, in any event let me know if you need any further info:)
Not sure if it's what you looking for, but judging from your starting conditions (Win C#/Linux C++) you may not need it to be totally language agnostic. You can implement such a parser yourself in C++ with all the desired features and then just use it in both C++ and C# projects - thus also bypassing the need to add external libraries.
On application design level, it would be (relatively) simple - you create a library which is buildable cross-platform and use it in both projects. The interface may be something simple like:
bool VerifyConstraint_int(int value, const char* constraint);
bool VerifyConstraint_double(double value, const char* constraint);
// etc
Such interface will be usable both in Linux C++ (by static or dynamic linking) and in Windows C# (using P/Invoke). You can have same codebase compiling on both platforms.
The parser (again, judging from what you've described in the question) may be pretty simple - a tree holding elements of types Variable and Expression which can be Evaluated with a given Variable value.
Example class definitions:
class Entity {public: virtual VARIANT Evaluate() = 0;} // boost::variant may be used typedef'd as VARIANT
class BinaryOperation: public Entity {
private:
Entity& left;
Entity& right;
enum Operation {PLUS,MINUS,EQUALS,AND,OR,GREATER_OR_EQUALS,LESS_OR_EQUALS};
public:
virtual VARIANT Evaluate() override; // Evaluates left and right operands and combines them
}
class Variable: public Entity {
private:
VARIANT value;
public:
virtual VARIANT Evaluate() override {return value;};
}
Or, you can just write validation code in C++ and use it both in C# and C++ applications :)
My personal choice would be Lua. The downside to any DSL is the learning curve of a new language and how to glue the code with the scripts but I've found Lua has lots of support from the user base and several good books to help you learn.
If you are after making somewhat generic code that a non programmer can inject rules for allowable input it's going to take some upfront work regardless of the route you take. I highly suggest not rolling your own because you'll likely find people wanting more features that an already made DSL will have.
If you are using Java then you can use the Object Graph Navigation Library.
It enables you to write java applications that can parse,compile and evaluate OGNL expressions.
OGNL expressions include basic java,C,C++,C# expressions.
You can compile an expression that uses some variables, and then evaluate that expression
for some given variables.
An easy way to achieve validation of expressions is to use Python's eval method. It can be used to evaluate expressions just like the one you wrote. Python's syntax is easy enough to learn for simple expressions and english-like. Your expression example is translated to:
(value >= 1 and value <= 3) or (value >= 10 and value <= 12)
Code evaluation provided by users might pose a security risk though as certain functions could be used to be executed on the host machine (such as the open function, to open a file). But the eval function takes extra arguments to restrict the allowed functions. Hence you can create a safe evaluation environment.
# Import math functions, and we'll use a few of them to create
# a list of safe functions from the math module to be used by eval.
from math import *
# A user-defined method won't be reachable in the evaluation, as long
# as we provide the list of allowed functions and vars to eval.
def dangerous_function(filename):
print open(filename).read()
# We're building the list of safe functions to use by eval:
safe_list = ['math','acos', 'asin', 'atan', 'atan2', 'ceil', 'cos', 'cosh', 'degrees', 'e', 'exp', 'fabs', 'floor', 'fmod', 'frexp', 'hypot', 'ldexp', 'log', 'log10', 'modf', 'pi', 'pow', 'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh']
safe_dict = dict([ (k, locals().get(k, None)) for k in safe_list ])
# Let's test the eval method with your example:
exp = "(value >= 1 and value <= 3) or (value >= 10 and value <= 12)"
safe_dict['value'] = 2
print "expression evaluation: ", eval(exp, {"__builtins__":None},safe_dict)
-> expression evaluation: True
# Test with a forbidden method, such as 'abs'
exp = raw_input("type an expression: ")
-> type an expression: (abs(-2) >= 1 and abs(-2) <= 3) or (abs(-2) >= 10 and abs(-2) <= 12)
print "expression evaluation: ", eval(exp, {"__builtins__":None},safe_dict)
-> expression evaluation:
-> Traceback (most recent call last):
-> File "<stdin>", line 1, in <module>
-> File "<string>", line 1, in <module>
-> NameError: name 'abs' is not defined
# Let's test it again, without any extra parameters to the eval method
# that would prevent its execution
print "expression evaluation: ", eval(exp)
-> expression evaluation: True
# Works fine without the safe dict! So the restrictions were active
# in the previous example..
# is odd?
def isodd(x): return bool(x & 1)
safe_dict['isodd'] = isodd
print "expression evaluation: ", eval("isodd(7)", {"__builtins__":None},safe_dict)
-> expression evaluation: True
print "expression evaluation: ", eval("isodd(42)", {"__builtins__":None},safe_dict)
-> expression evaluation: False
# A bit more complex this time, let's ask the user a function:
user_func = raw_input("type a function: y = ")
-> type a function: y = exp(x)
# Let's test it:
for x in range(1,10):
# add x in the safe dict
safe_dict['x']=x
print "x = ", x , ", y = ", eval(user_func,{"__builtins__":None},safe_dict)
-> x = 1 , y = 2.71828182846
-> x = 2 , y = 7.38905609893
-> x = 3 , y = 20.0855369232
-> x = 4 , y = 54.5981500331
-> x = 5 , y = 148.413159103
-> x = 6 , y = 403.428793493
-> x = 7 , y = 1096.63315843
-> x = 8 , y = 2980.95798704
-> x = 9 , y = 8103.08392758
So you can control the allowed functions that should be used by the eval method, and have a sandbox environment that can evaluate expressions.
This is what we used in a previous project I worked in. We used Python expressions in custom Eclipse IDE plug-ins, using Jython to run in the JVM. You could do the same with IronPython to run in the CLR.
The examples I used in part inspired / copied from the Lybniz project explanation on how to run a safe Python eval environment. Read it for more details!
You might want to look at Regular-Expressions or RegEx. It's proven and been around for a long time. There's a regex library all the major programming/script languages out there.
Libraries:
C++: what regex library should I use?
C# Regex Class
Usage
Regex Email validation
Regex to validate date format dd/mm/yyyy
I am trying to transfer money from one "account" to another:
puts ("\nTransfer how much?")
require 'bigdecimal'
amount = gets.chomp
amount = BigDecimal(amount) #<--Does BigDecimal have to work with strings???
puts ("\nTransfer to which account?")
acct_to = gets.chomp.to_i #<--Accounts are numbered previously for the user.
acct_to = acct_to - 1 #<--Since the account numbers are stored in an array...
#having a problem with #{amount} in the string below. It is showing something like
#1.2E0???
puts ("\nTransfer #{amount} from #{names[acct_from]} to #{names[acct_to]}? [1 - yes] / [2 - no]")
#Makes transfer==============================
e = gets.chomp.to_i
if e == 1
puts ("\nTransferring!")
sum1 = 0
sum2 = 0
sum1 = BigDecimal(ac[names[acct_from]].to_s) #<-- ac is a hash
sum2 = BigDecimal(ac[names[acct_to]].to_s)
ac[names[acct_from]] = sum1 - amount
ac[names[acct_to]] = sum2 + amount
puts ("\n#{names[acct_from]}'s new balance is #{ac[names[acct_from]]}.")
puts ("\n#{names[acct_to]}'s new balance is #{ac[names[acct_to]]}.")
end
end
Ok, I have this working really well with numbers operating as floats; however, as you know, floats are causing problems.
Please help me get an introductory grasp of how bigdecimal works.
Also, if you are really awesome, help me get it to work in this specific situation.
First, if it is working with floats, you can get it to work with BigDecimal as well, and you should, because of the obvious reasons.
So, to answer your first question in the comments to your code: Yes, BigDecimal instantiation has to work with strings, the reason is quite obvious: Stringified number values are not prone to any inaccuracies and do not share the limits of float representation:
# Think of this number
float = 23.12323423142342348273498721348923748712340982137490823714089374
# Ruby will truncate its precision to 17, because Float's are not made for precise, but for fast calculation
float #=> 23.123234231423424
# Now, if BigDecimal would initialize with a float value, the precision would get lost on the way, too. Therefore, BigDecimal needs strings
big_decimal_from_float = BigDecimal.new(23.12323423142342348273498721348923748712340982137490823714089374.to_s)
big_decimal_from_string = BigDecimal.new("23.12323423142342348273498721348923748712340982137490823714089374")
# Now you'll see that the BigDecimal initialized "with a float value" will have lost some precision
To answer your second question, 1.2E0 is just Scientific Notation for 1.2. BigDecimal always uses Scientific Notation since it is intented for use in really precise calculations used in science and financial math.
To comment on your example, using BigDecimal is surely the right way to go, but you have to use it throughout and store your values accordingly. That means that if you write to an SQL database, you will have to use a decimal format with the right precision. Also, all instantiations thereform have to be instances of BigDecimal and never Float. One float in your entire finance application can rain on your parade if you intend to do financial math with really tiny fractures or high values.
To relieve you of some of the pitfalls of money handling, have a look at the Exchange Gem. I wrote it to have a way of representing money in a ruby application using BigDecimal and ISO4217 compatible instantiation. It may help you handling money throughout an application and avoid some of the pitfalls involved.
I suggesting you to use this gem: github/RubyMoney/money
Read a bit more about it. It is working out-of-the-box. It use nor float, nor BigDecimal, but just integers. So no precision loss at all.
my goal is to write an algorithm in Mathematica that will search for stocks whose current price is trading below or above 2 Standard Deviations of the Mean. I literally started to learn the program yesterday but I have been scouring the internet ever since for help. I have the code, but I am getting errors along the way. can someone help me?
Below is my current code
Today = Date[]
StartDate = Today-{0,3,0,0,0,0}
NYSEMem = FinancialData["NYSE","Members"]
CurrentPrice = FinancialData[NYSEMem,"Price"]
HistoricalPrice = FinancialData[NYSEMem,{{StartDate},{Today}}]
StandardDeviation$ = StandardDeviation[HistoricalPrice]
MeanPrice = Mean[HistoricalData]
SellSignal = [MeanPrice]-[StandardDeviation$]*2
BuySignal = [MeanPrice]+[StandardDeviation$]*2
If[CurrentPrice>SellSignal,"Sell",False]
If[CurrenPrice<BuySignal,"Buy",False]
Very brave to jump straight in the deep waters, but I'd suggest trying to learn the basics first. You say you've been "scouring the internet for help" but did you tried Mathematica's on-board documentation center? It has thousands pages of help, just one key press away.
Anyway as to your code, a few tips:
Don't end a variable with a $. Though
not wrong in principle, they are used
for system variables
The line SellSignal =
[MeanPrice]-[StandardDeviation$]*2, and the one following it contain function call brackets without corresponding function names. You probably don't intend to have these brackets there
The ,False part in If[CurrentPrice>SellSignal,"Sell",False] and the next line is unnecessary and can be deleted here
The earlier date calculation can be better done with the dedicated DatePlus function, which takes things like leap years etc. into account
You probably don't want to see all the output of all the lines. You may suppress output using ';' (which also acts to separate compound statements)
An asterisk for multiplication is unnecessary. A space will do, just as in math. a*b, a b, a 2, 2 a and 2a (without a space) are all correct multiplications.
The data you receive from some of the calls include both prices and dates. You are also trying to average the dates and find their standard deviation.
Though it is allowed to start a variable with an uppercase letter you better avoid doing that in order to prevent using Mathematica's own keywords (which all start with an uppercase letter).
I don't think your buy and sell signals are very smart. You might think of selling when prices are historically high, but you're doing it when they are above the historic low watermark.
Same for buying. Also, when current price is between your two signals the program provides conflicting advice.
You need a construction to repeat the calculations for every NYSE member
Some very very basic code to get you started:
StartDate = DatePlus[Date[], {-3, "Month"}];
NYSEMem = Select[FinancialData["NYSE", "Members"], (\[Not] StringMatchQ[#, ___ ~~
"^" ~~ ___] &)]; (* Throw away indices *)
Do[
currentPrice = Check[FinancialData[stock, "Price"], $Failed];
historicalPrice =
Check[FinancialData[stock, {StartDate, Date[]}], $Failed];
If[currentPrice == $Failed || historicalPrice == $Failed ||
currentPrice == Missing["NotAvailable"] ||
historicalPrice == Missing["NotAvailable"],
Continue[]]; (* Shamefully inadequate error handling *)
standardDeviationPrice = StandardDeviation[historicalPrice[[All, 2]]];
meanPrice = Mean[historicalPrice[[All, 2]]];
(* Mean of the second column of the data matrix *)
sellSignal = meanPrice + 2 standardDeviationPrice;
(* swapped + and - in these two lines, plug your own method here *)
buySignal = meanPrice - 2 standardDeviationPrice;
Print[stock, ": ",
If[currentPrice > sellSignal, "Sell",
If[currentPrice < buySignal, "Buy", "Neutral"]]];
, {stock, NYSEMem}
]
Please note that Stackoverflow is intended for people who have faithfully tried to do their best to do some research in the problems they encounter. I have the feeling that you don't really meet this criterion. My urgent request is: read some basic introductory text about Mathematica (for instance Getting started and Core Language Overview).
Here you have your program running:
Today = Date[];
StartDate = Today - {0, 3, 0, 0, 0, 0};
NYSEMem = FinancialData["NYSE", "Members"];
NYSEMem = NYSEMem[[1000 ;; 1001]];
CurrentPrice = FinancialData[#, "Price"] & /# NYSEMem;
HistoricalPrice = FinancialData[#, {StartDate, Today}] & /# NYSEMem;
StandardDeviation$ = StandardDeviation[#[[All, 2]]] & /# HistoricalPrice;
MeanPrice = Mean[#[[All, 2]]] & /# HistoricalPrice;
SellSignal = MeanPrice - StandardDeviation$*2
BuySignal = MeanPrice + StandardDeviation$*2
Do[
If[CurrentPrice[[i]] > SellSignal[[i]], Print["Sell ", NYSEMem[[i]]]];
If[CurrentPrice[[i]] < BuySignal[[i]], Print["Buy ", NYSEMem[[i]]]],
{i, 2}]
But please note I modified just the minimum to get it running without using idioms. This is not, by any standard, a good program. I did it just to let you play a little with it and learn some constructs.
HTH!