Let's try this :
float test = 3.56;
float roundedVal = round(test * 10.0f) / 10.0f;
NSLog (#"%f", roundedVal);
Why does tthe NSLog shows 3.600000, and the debugger 3.5999999 ?
What is the correct value I can count on ?
A float is not very precise and you can't expect them to be displayed accurately from one "view" to the next depending on a variety of factors. Read What Every Computer Scientist Should Know About Floating-Point Arithmetic.
Boils down to this: use double if you really want precision.
The correct value is the debugger. You can see this with a simplified version of the code above:
float test = 3.5999999;
NSLog (#"%f", test);
In this case, you get the same results as what you mentioned above: the log states 3.600000 and the debugger states it is 3.5999999. In all cases, the debugger has the correcrt value over an NSLog. When we dig a bit deeper, we can see that NSLog is slightly massaging the float value.
In reality - you should probably use a double here to maintain the precision you are looking for.
There is no "correct value". The expected value "3.6" does not have an exact binary representation. Therefore, the value the computer uses internally is an approximation. When printing the value, it needs to convert that back to decimal. In your case, the binary representation is likely slightly less than 3.6. NSLog happens to round it up, while the debugger rounds it down.
Using double instead of float, like the accepted answer says, does not change anything in this.
Related
I'm trying to make a "complex calculation" in a freemarker template. The calculation calls for a modulus call. At first I kept simplifying my code, and finally just hardcoding values to try and figure out the cause, but it seems that freemarker's mod function ALWAYS returns int values?? but they're not even rounded, they're truncated. I need them rounded (.5 up would work). My most simplified code that does NOT work is this:
<#assign p_year_m1 = (2503.638 % 7.00)?float>
the longer version is:
<#assign p_year_m1 = ((bdy_m1 + (bdy_m1/4.0)-(bdy_m1/100.0)+(bdy_m1/400.0)))%7>
So the correct answer should be 4.638, rounds up to 5, but I'm getting 4.00 even if I wrap ?string[0.00] around it.
What am I missing? Surely there has to be a way to make this work! If not, I guess I'd need help constructing a work around to get similar results.
Thanks.
PS - the var, p_year_m1 is not intialized before this, so I believe if I try to store a float in there it should hold a float.
This is the calculation I'm trying to replicate
I can confirm that % gives the remainder of integer division. It casts (truncates) both arguments to integers, and then calculates the remainder. It's weird for sure, but can't be changed because of backward compatibility. The main application is calculating "zebra tables" and such, where it's might as well desirable, I don't know... it had just get in like that in the early times of the project, and then stuck.
I've profiled my program with Valgrind and Callgrind and found that most of the time is spent in the nearbyint$fenv_access_off function.
I've found that it's a LLVM intrinsic, but which Rust language construct uses it? How can I avoid it?
Doing a search for nearbyint finds the related symbols nearbyintf32 and nearbyintf64. These are documented as returning the nearest integer to a floating point value. However, there appears to be no calls to that specific function.
fenv_access_off appears to be an OS X specific aspect of the math library.
The other thing in your trace is round. I can believe that round could use nearbyint. I also don't see any cases of round in the standard library that seem like they would occur in a tight loop.
Beyond this, anything is pure guessing.
I've reproduced it with:
fn main() {
let data:Vec<_> = (0..999999).map(|x|{
(x as f64).powf(2.2).round() as u8
}).collect();
}
so it seems as u8 is implemented using nearbyint.
It's the same speed as C uchar = round(pow(i, 2.2)), so I'll have to replace it with a good'ol lookup table…
i have the below actionscript and looking for the equivalent ruby on rails code to do the same job.
where dbwall and db_wall = 50
var tb= Math.pow((1/10),(dbwall)/(10));
Currently i have used:
#tb = ((1/10) ** (db_wall)/10)
and get 0.0, which is not what i need:
however in actionscript the actual answer to this is 0.00001
Is it just a case of the correct formula but not showing decimal places? or is there more to it than that?
Integer division in Ruby returns an integer (like C) so 1/10 equals 0. To get Float division, explicitly make one of the operands a Float e.g. 1.0/10
You don't need to include a few of the brackets too.
#tb = ((1/10) ** db_wall/10)
Would simply work. I wouldn't make this variable accessible to the view by using # if you carry on, just take that off and use it in future calculations.
What Max has answered would be the best answer to this simple problem.
Just got a "FloatDomainError" in my app, with the message "NaN" where the server would ordinarily describe the offense. The server error page points me to these lines (well, the first one):
n = ((self.weight * (c.percent)/100) / c.package_weight).to_i
n.times do
The problem seems to be with the .to_i, which is meant to convert what totals to 8.35 into an integer so I can do n.times do.
The problem is, if I take to_i away, I get an error telling me, logically enough, that there's no method "times" for the float 8.35. But I don't get why it's not letting me round off 8.35 to an integer. I've tried .floor too, but I get the FloatDomainError.
Ideas on how to round this off so I can get it to work?
Thanks!
This error means that you tried to convert a NaN value to int. NaN stands for "Not a number". That probably meants that your calculation is flawed and not result in 8.35. Maybe the value of c.package_weight is zero and you divide with it.
this is my first post here. I started using Ruby just 2 days ago and think it is an amazing language, however I have been getting stuck. My problem is I am wanting to calculate the event horizon of a black hole given an input defined in the code as "m" This will then be put into a calculation and the size then printed out to the screen. I did need it to be in binary and thats where I am having the issue.
Here is my code so far.
#Event Horizon Calculation Program
G = 6.67*10**-11
m = 20
C = 200000
R = G*m/(C**2)
puts "Here is the result in Binary."
R.to_i(2)
puts R
Now I do realise that the number are not accurate, that dosen't matter at the moment. I just need the function to work.
Thankyou,
Ross.
Your post is not even in a format of asking a question, but guessing from what you wrote, it seems that you are asking how to change your code so that it accepts an input to m and outputs the result. My answer is based on this assumption.
In order to take an input, use the 'gets' method. So, you may want to replace your 'm = 20' line with:
m = gets.to_f
'gets' accepts an input as a string, so you need to convert it to a numeric. to_f changes a string into a float. You can use to_i instead if you want an integer.
You have a line 'R.to_i(2)', and it seems like you want to output this, but you have two problems here. First of all, whatever that creates, it is only creating something in that position, and does not change the value of R, so, in effect, it actually does nothing. Second, ruby can accept numerals in source code written in different bases such decimal, binary, hex, etc., but it only has one internal representation, and you cannot output a numeral in binary. For your purpose, you need to convert it to a string that corresponds to a binary expression. For that, use the 'to_s' method. In fact, the 'to_i' method does not take an argument. Delete your line 'R.to_i(s)', and replace the line 'puts R' with:
puts R.to_s(2)