In my python script, I am doing multiplication on some values in a CSV file. I am trying to use argparse so that I can pass the (float) values (which I want to multiply the CSV values with), something like
python3 script.py filename.csv 0.5 2 3.5
I tried doing the argparse as following:
ap = argparse.ArgumentParser()
ap.add_argument("-m", "--multiplier", required = True, help = "number to multiply with x,y,z values")
args = vars(ap.parse_args())
and then using args["multiplier"] where I am doing the multiplication in the script.
I don't know if I am making a mistake in the code, or while calling the script
I am not sure how you are calling your script.
import argparse
ap = argparse.ArgumentParser()
ap.add_argument("-m", "--multiplier", required = True, help = "number to multiply with x,y,z values")
args = vars(ap.parse_args())
print(float(args['multiplier']))
The code does work as expected. Maybe you are calling it wrong. To call your code :
your_prompt$ python3 try.py -m 2
2.0
In your code you are specifying an argument -m. So to pass your multiplier to the script you need to add a -m parameter.
Related
What will be output of the code:
def gfg(x,l=[]):
for i in range(x):
l.append(i*i)
print(l)
What is the output of following function calls-
gfg(3,[3,2,1])
gfg(3)
First of all welcome to stackoverflow! For the following code it's probably best to just try to run your code in a python shell and see the output. But I can provide some explanation as to why you get the output that you do.
Your code was as follows:
def gfg(x, l=[]):
for i in range(x):
l.append(i*i)
print(l)
And we are trying to run this function on the following input:
gfg(3, [3,2,1]) and gfg(3).
The output that we get is as follows:
gfg(3, [3,2,1]) = [3,2,1,0,1,4]
gfg(3) = [0,1,4]
This is because python uses something called default arguments that are used when we do not pass in the required number of arguments into a function. So in the first case when we print out l we get that 0,1,4 is appended onto the end of the existing parameter l that is passed into the function ([3,2,1]. And when we do not pass in an argument for l the output is just the list [0,1,4].
I'm writing a custom objective to train a Keras (with TensorFlow backend) model but I need to debug some intermediate computation. For simplicity, let's say I have:
def custom_loss(y_pred, y_true):
diff = y_pred - y_true
return K.square(diff)
I could not find an easy way to access, for example, the intermediate variable diff or its shape during training. In this simple example, I know that I could return diff to print its values, but my actual loss is more complex and I can't return intermediate values without getting compiling errors.
Is there an easy way to debug intermediate variables in Keras?
This is not something that is solved in Keras as far as I know, so you have to resort to backend-specific functionality. Both Theano and TensorFlow have Print nodes that are identity nodes (i.e., they return the input node) and have the side-effect of printing the input (or some tensor of the input).
Example for Theano:
diff = y_pred - y_true
diff = theano.printing.Print('shape of diff', attrs=['shape'])(diff)
return K.square(diff)
Example for TensorFlow:
diff = y_pred - y_true
diff = tf.Print(diff, [tf.shape(diff)])
return K.square(diff)
Note that this only works for intermediate values. Keras expects tensors that are passed to other layers to have specific attributes such as _keras_shape. Values processed by the backend, i.e. through Print, usually do not have that attribute. To solve this, you can wrap debug statements in a Lambda layer for example.
In TensorFlow 2, you can now add IDE breakpoints in the TensorFlow Keras models/layers/losses, including when using the fit, evaluate, and predict methods. However, you must add model.run_eagerly = True after calling model.compile() for the values of the tensor to be available in the debugger at the breakpoint. For example,
import tensorflow as tf
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam
def custom_loss(y_pred, y_true):
diff = y_pred - y_true
return tf.keras.backend.square(diff) # Breakpoint in IDE here. =====
class SimpleModel(Model):
def __init__(self):
super().__init__()
self.dense0 = Dense(2)
self.dense1 = Dense(1)
def call(self, inputs):
z = self.dense0(inputs)
z = self.dense1(z)
return z
x = tf.convert_to_tensor([[1, 2, 3], [4, 5, 6]], dtype=tf.float32)
y = tf.convert_to_tensor([0, 1], dtype=tf.float32)
model0 = SimpleModel()
model0.run_eagerly = True
model0.compile(optimizer=Adam(), loss=custom_loss)
y0 = model0.fit(x, y, epochs=1) # Values of diff *not* shown at breakpoint. =====
model1 = SimpleModel()
model1.compile(optimizer=Adam(), loss=custom_loss)
model1.run_eagerly = True
y1 = model1.fit(x, y, epochs=1) # Values of diff shown at breakpoint. =====
This also works for debugging the outputs of intermediate network layers (for example, adding the breakpoint in the call of the SimpleModel).
Note: this was tested in TensorFlow 2.0.0-rc0.
In TensorFlow 2.0, you can use tf.print and print anything inside the definition of your loss function. You can also do something like tf.print("my_intermediate_tensor =", my_intermediate_tensor), i.e. with a message, similar to Python's print. However, you may need to decorate your loss function with #tf.function to actually see the results of the tf.print.
In tensorflow CIFAR-10 tutorial in cifar10_inputs.py line 174 it is said you should randomize the order of the operations random_contrast and random_brightness for better data augmentation.
To do so the first thing I think of is drawing a random variable from the uniform distribution between 0 and 1 : p_order. And do:
if p_order>0.5:
distorted_image=tf.image.random_contrast(image)
distorted_image=tf.image.random_brightness(distorted_image)
else:
distorted_image=tf.image.random_brightness(image)
distorted_image=tf.image.random_contrast(distorted_image)
However there are two possible options for getting p_order:
1) Using numpy which disatisfies me as I wanted pure TF and that TF discourages its user to mix numpy and tensorflow
2) Using TF, however as p_order can only be evaluated in a tf.Session()
I do not really know if I should do:
with tf.Session() as sess2:
p_order_tensor=tf.random_uniform([1,],0.,1.)
p_order=float(p_order_tensor.eval())
All those operations are inside the body of a function and are run from another script which has a different session/graph. Or I could pass the graph from the other script as an argument to this function but I am confused.
Even the fact that tensorflow functions like this one or inference for example seem to define the graph in a global fashion without explicitly returning it as an output is a bit hard to understand for me.
You can use tf.cond(pred, fn1, fn2, name=None) (see doc).
This function allows you to use the boolean value of pred inside the TensorFlow graph (no need to call self.eval() or sess.run(), hence no need of a Session).
Here is an example of how to use it:
def fn1():
distorted_image=tf.image.random_contrast(image)
distorted_image=tf.image.random_brightness(distorted_image)
return distorted_image
def fn2():
distorted_image=tf.image.random_brightness(image)
distorted_image=tf.image.random_contrast(distorted_image)
return distorted_image
# Uniform variable in [0,1)
p_order = tf.random_uniform(shape=[], minval=0., maxval=1., dtype=tf.float32)
pred = tf.less(p_order, 0.5)
distorted_image = tf.cond(pred, fn1, fn2)
I'm trying to create a random number generator in Lua. I found out that I can just use math.random(1,100) to randomize a number between 1 and 100 and that should be sufficient.
But I don't really understand how to use the randomize number as variables in the script.
Tried this but of course it didn't work.
$randomCorr = math.random(1,100);
http.request_batch({
{"POST", "https://store.thestore.com/priceAndOrder/selectProduct", headers={["Content-Type"]="application/json;charset=UTF-8"}, data="{\"ChoosenPhoneModelId\":4,\"PricePlanId\":\"phone\",\"CorrelationId\":\"$randomCorr\",\"DeliveryTime\":\"1 vecka\",\"$$hashKey\":\"006\"},\"ChoosenAmortization\":{\"AmortizationLength\":0,\"ChoosenDataPackage\":{\"Description\":\"6 GB\",\"PricePerMountInKr\":245,\"DataAmountInGb\":6,\"$$hashKey\":\"00W\"},\"ChoosenPriceplan\":{\"IsPostpaid\":true,\"Title\":\"Fastpris\",\"Description\":\"Fasta kostnader till fast pris\",\"MonthlyAmount\":0,\"AvailiableDataPackages\":null,\"SubscriptionBinding\":0,\"$$hashKey\":\"00K\"}}", auto_decompress=true},
{"GET", "https://store.thestore.com/api/checkout/getproduct?correlationId=$randomCorr", auto_decompress=true},
})
In Lua, you can not start a variable name with $. This is where your main issue is at. Once the $ is removed from your code, we can easily see how to refer to variables in Lua.
randomCorr = math.random(100)
print("The random number:", randomCorr)
randomCorr = math.random(100)
print("New Random Number:", randomCorr)
Also, concatenation does not work the way you are implying it into your Http array. You have to concatenate the value in using .. in Lua
Take a look at the following example:
ran = math.random(100)
data = "{\""..ran.."\"}"
print(data)
--{"14"}
The same logic can be implied into your code:
data="{\"ChoosenPhoneModelId\":4,\"PricePlanId\":\"phone\",\"CorrelationId\":\""..randomCorr.."\",\"DeliveryTime\":\"1 vecka\",\"$$hashKey\":\"006\"},\"ChoosenAmortization\":{\"AmortizationLength\":0,\"ChoosenDataPackage\":{\"Description\":\"6 GB\",\"PricePerMountInKr\":245,\"DataAmountInGb\":6,\"$$hashKey\":\"00W\"},\"ChoosenPriceplan\":{\"IsPostpaid\":true,\"Title\":\"Fastpris\",\"Description\":\"Fasta kostnader till fast pris\",\"MonthlyAmount\":0,\"AvailiableDataPackages\":null,\"SubscriptionBinding\":0,\"$$hashKey\":\"00K\"}}"
Or you can format the value in using one of the methods provided by the string library
Take a look at the following example:
ran = math.random(100)
data = "{%q}"
print(string.format(data,ran))
--{"59"}
The %q specifier will take whatever you put as input, and safely surround it with quotations
The same logic can be applied to your Http Data.
Here is a corrected version of the code snippet:
local randomCorr = math.random(1,100)
http.request_batch({
{"POST", "https://store.thestore.com/priceAndOrder/selectProduct", headers={["Content-Type"]="application/json;charset=UTF-8"}, data="{\"ChoosenPhoneModelId\":4,\"PricePlanId\":\"phone\",\"CorrelationId\":\"" .. randomCorr .. "\",\"DeliveryTime\":\"1 vecka\",\"$$hashKey\":\"006\"},\"ChoosenAmortization\":{\"AmortizationLength\":0,\"ChoosenDataPackage\":{\"Description\":\"6 GB\",\"PricePerMountInKr\":245,\"DataAmountInGb\":6,\"$$hashKey\":\"00W\"},\"ChoosenPriceplan\":{\"IsPostpaid\":true,\"Title\":\"Fastpris\",\"Description\":\"Fasta kostnader till fast pris\",\"MonthlyAmount\":0,\"AvailiableDataPackages\":null,\"SubscriptionBinding\":0,\"$$hashKey\":\"00K\"}}", auto_decompress=true},
{"GET", "https://store.thestore.com/api/checkout/getproduct?correlationId=" .. randomCorr, auto_decompress=true},
})
There is something called $$hashKey also, in the quoted string. Not sure if that is supposed to be referencing a variable or not. If it is, it also needs to be concatenated into the resulting string, using the .. operator (just like with the randomCorr variable).
I'm not sure if this is a PyMC3 question or a Theano question. I've used PyMC2 for a long time to fit a cosmology to supernova data. This requires some messy integrals (see i.e. http://arxiv.org/abs/astroph/9905116 )
So I use a package in python called Cosmolopy to do the integration and for some other convenience functions. Whereas this used to work fine with PyMC2, with the reliance on theano in PyMC3, I can't figure out if there is even a way to use Cosmolopy.
Here is some example code of my current understanding of how to build a model in PyMC3
import numpy as np
import pymc as pm
import cosmolopy as cp
# generate some redshifts
nSNe = 100
z = np.random.uniform( low=0.0, high=1.0, size=nSNe )
# set cosmology and simulate some distance moduli and errors
cosmo = cp.fidcosmo
muSN = cp.magnitudes.distance_modulus( z, **cosmo ) + np.random.normal( loc=0, scale=0.15, size=nSNe )
muSN_err = np.random.uniform(low=0.1, high=0.3, size=nSNe)
# pymc model
with pm.Model() as model:
# omega matter is the free parameter in this simple example
omega_matter = pm.Uniform( 'omega_matter', lower=0.0, upper=1.0 )
# the cosmology as a function of omega_matter
cosmo['omega_M_0'] = omega_matter
cosmo['omega_lambda_0'] = 1.0 - omega_matter
mu_fit = cp.magnitudes.distance_modulus( z, **cosmo )
# what should be fit by the MCMC
snr = pm.Normal( 'snr', mu = mu_fit, sd = muSN_err, observed = muSN )
This code crashes because Cosmolopy expects a float for omega_matter but receives a theano.TensorVariable instead.
So the question is two-fold:
Am I just missing something syntactically with PyMC3 that would allow me to do this (possibly because I am still stuck somehow on PyMC2 model-building)?
If not 1, then do I need to find a way to do the integrals in theano?
I don't know well PyMC3, but I know well Theano. Theano use symbolic compiler and TensorVariable are such symbolic variable. You need to compile and execute the function to get a value out of it. I don't know where to do this in PyMC3. A fast thing to try that will work if the variable depend only on constant and shared variable is to do this call::
the_tensor_variable.eval()
This will compile the function and suppose it don't take any variable input and if it compile, it will run it and return the value.
I think one possible solution would be to write a custom Theano Op following the instructions at http://deeplearning.net/software/theano/extending/
I would write a pure Python op without support for gradient computation, in which you would only have to implement the make_node() and perform() methods.