In submitting my AMPL formulation to the NEOS server, why does the below codeline using the `setof' operator work fine in a model file but not in a data file?
set X := setof {indices in Y} subsetofindices
The NEOS server requires the code be separated into a .mod , .dat, and script file separately. I've seen references to use let X := ... in the script file. What is the difference between these files, and what to put where?
Example
Using AMPL's steelT.x files, this first block of 2 files works
# steelT.mod
set DUMMYPROD;
set PROD := setof {i in DUMMYPROD} i; # products
# steelT.dat
data;
set DUMMYPROD := bands coils;
Whereas the following does not work:
# steelT.mod
set DUMMYPROD;
set PROD; # products
# steelT.dat
data;
set DUMMYPROD := bands coils;
set PROD := setof {i in DUMMYPROD} i;
It returns the error
expected ; ( : or symbol
context: set PROD := setof >>> { <<< i in DUMMYPROD} i;
Broader question
Generally, what belongs in the model file, data file and script file? (Also for param and other definitions) What is the order in which these are processed? I may be missing some basics as I read the AMPL guide where I did not find this (I never took an Optimization course).
Bonus question about NEOS server
"New" Contraint programming logic in AMPL includes conditional and logical operators for entire constraints. I don't think these are available in NEOS. Or are they (because they didn't work for me)? Is there a simple workaround?
Thanks!
AMPL data format doesn't support expressions, so you should either use setof {i in DUMMYPROD} i in the declaration (as in your first example) or in the AMPL script:
let PROD := setof {i in DUMMYPROD} i;
See also Chapter 9. Specifying Data of the AMPL book for more details on the AMPL data format.
As for the constraint programming (CP) features, I don't think there are any CP solvers on NEOS. You can try contacting NEOS Support regarding this.
Related
I need to develop a script to run a simple OLS on multiple csv files stored in the same folder.
All have the same column names and regression will always be based upon the same columns ("x_var" and "y_var").
The below code is used to read in the csvs and rename them.
## Read in files from folder
file.List <- list.files(pattern = "*.csv")
for(i in 1:length(file.List))
{
assign(paste(gsub(".csv","", file.List[i])), read.csv(file.List[i]))
}
However, after this [very initial stage!] I've got a bit lost........
Each dataframe has 7 identical columns. a, b, c, d, x_var, e, y_var.....
I need to run a simple OLS using lm(x_car ~ y_var, data = dataframes) and plot the result on each dataframe and assumed a 'for loop' would be the best option, but am not too sure of how to do so....
After each regression is run I want it to extract the coefficients/R2 etc into a csv and save the plot separately.......
Tried below, but have gone very wrong [and not working at all];
list <- list(gsub(" finalSIRTAnalysis.csv","", file.List))
for(i in length(file.List))
{
lm(x_var ~ y_var, data = [i])
}
Can't even make a start on this........and need some advice, if anyone has any good ideas (such as creating an external function first.....)
I am not sure if the function lm is available to compute the results using multiple variable sources. Try merging the database. I have have a similar issue because I have 5k files and is computationally impossible to merge them all. But maybe this answer can help you.
https://stackoverflow.com/a/63770065/14744492
I am trying to understand the query plan of MonetDB.
Is there a documentation anywhere where I can find what each instruction stays for?
If not, can anybody tell me what are returning
sql.projectdelta(X_15,X_23,X_25,r1_30,X_27)
and
sql.subdelta(X_246,X_4,X_10,X_247,X_249), for example?
In my query I am sorting the result by two attributes (e.g., by A,B). Can you tell me why the second sort has more parameters than the first?
(X_29,r1_36,r2_36) := algebra.subsort(X_28,false,false);
(X_33,r1_40,r2_40) := algebra.subsort(X_22,r1_36,r2_36,false,false);
Is algebra.subsort returning (oid, columnType) pairs, or just oid?
Thank you!!
Understanding output of the explain SQL statement requires knowledge of the MonetDB Assembly-like Language (MAL).
Concerning functions sql.projectdelta, sql.subdelta, and algebra.subsort, you'll find their signature and a (brief) description in the monetdb lib folder. Ex :
[MonetDB_install_folder]\MonetDB5\lib\monetdb5\sql.mal for all sql functions
[MonetDB_install_folder]\MonetDB5\lib\monetdb5\algebra.mal for all algebra functions
Concerning the different number of parameters for algebra.subsort :
(X_29,r1_36,r2_36) := algebra.subsort(X_28,false,false);
is described as :
Returns a copy of the BAT sorted on tail values, a BAT that specifies
how the input was reordered, and a BAT with group information.
The input and output are (must be) dense headed.
The order is descending if the reverse bit is set.
This is a stable sort if the stable bit is set.
(X_33,r1_40,r2_40) := algebra.subsort(X_22,r1_36,r2_36,false,false);
is described as:
Returns a copy of the BAT sorted on tail values, a BAT that specifies
how the input was reordered, and a BAT with group information.
The input and output are (must be) dense headed.
The order is descending if the reverse bit is set.
This is a stable sort if the stable bit is set.
MAL functions can be overloaded bassed on their return value. algebra.subsort can return 1, 2 or 3 values depending on what you're asking for. Checl algebra.mal for the different possibilities.
I just started using SPSS, there is a option of Select cases that I was trying in SPSS, and later on finding frequency based on that filter.
For Eg:
Suppose Q1 has 12 parts, Q1_1 Q1_2 Q1_3 Q1_4 Q1_5 Q1_6 Q1_7 Q1_8 Q1_9 Q1_10 Q1_11 Q1_12
I want to see data in these variables based on a condition that I used in select cases. Now when I try to see frequencies of these variables based on the filter, only 4 out of 12 satisfy has data.
Now my question is can I hide rest 8 and show only 4 with data on my output window.
It's not entirely clear what you are trying to describe however reading between the lines, I'm guessing you are trying to delete tables generated from FREQUENCIES which may happen to be empty (likely due to a filter applied but perhaps not necessarily either)
You could do this with SPSS Scripting but avoiding that, you may want to explore using CTABLES, which though may not be in the exact same format as FREQUENCY table output it will still none the less retrieve the same information.
Solution below. Assumes Python Integration with SPSS SELECT VARIABLES installed and of course the CTABLE add-on module.
/****** Simulate example data ******/.
input program.
loop #j = 1 to 100.
compute ID=#j.
vector Q(12).
loop #i = 1 to 12.
do if #j<51 and #i<9.
compute Q(#i) = $sysmis.
else.
compute Q(#i) = trunc(rv.uniform(1,5)).
end if.
end loop.
end case.
end loop.
end file.
end input program.
execute.
/************************************/.
/* frequencies without filtering applied */.
freq q1 to q12.
/* frequencies WITH filtering applied */.
/* Empty table here shoult be removed */.
temp.
select if (ID<51).
freq q1 to q12.
spssinc select variables macroname="!Qp" /properties pattern = "^Q\d+$"/options separator="+" order=file.
spssinc select variables macroname="!Qs" /properties pattern = "^Q\d+$"/options separator=" " order=file.
temp.
select if (ID<51).
ctables /table (!Qp)[c][count colpct]
/categories variables=!Qs empty=exclude.
Note if you had assess empty variables at a total level then there is a function in spssaux2 (spssaux2.FindEmptyVars) which could help you find the empty variables and then you could build the syntax to exclude these and so contain the variables with only valid responses and then run FREQUENCIES. But I don't think spssaux2.FindEmptyVars will honor any filtering.
I am attempting to use fortran to write out a comma-delimited file for import into another commercial package. The issue is that I have an unknown number of data columns. My output needs to look like this:
a_string,a_float,a_different_float,float_array_elem1,float_array_elem2,...,float_array_elemn
which would result in something that might look like this:
L1080,546876.23,4325678.21,300.2,150.125,...,0.125
L1090,563245.1,2356345.21,27.1245,...,0.00983
I have three issues. One, I would prefer the elements to be tightly grouped (variable column width), two, I do not know how to define a variable number of array elements in the format statement, and three, the array elements can span a large range--maybe 12 orders of magnitude. The following code conceptually does what I want, but the variable 'n' and the lack of column-width definition throws an error (of course):
WRITE(50,900) linenames(ii),loc(ii,1:2),recon(ii,1:n)
900 FORMAT(A,',',F,',',F,n(',',F))
(I should note that n is fixed at run-time.) The write statement does what I want it to when I do WRITE(50,*), except that it's width-delimited.
I think this thread almost answered my question, but I got quite confused: SO. Right now I have a shell script with awk fixing the issue, but that solution is...inelegant. I could do some manipulation to make the output a string, and then just write it, but I would rather like to avoid that option if at all possible.
I'm doing this in Fortran 90 but I like to try to keep my code as backwards-compatible as possible.
the format close to what you want is f0.3, this will give no spaces and a fixed number of decimal places. I think if you want to also lop off trailing zeros you'll need to do a good bit of work.
The 'n' in your write statement can be larger than the number of data values, so one (old school) approach is to put a big number there, eg 100000. Modern fortran does have some syntax to specify indefinite repeat, i'm sure someone will offer that up.
----edit
the unlimited repeat is as you might guess an asterisk..and is evideltly "brand new" in f2008
In order to make sure that no space occurs between the entries in your line, you can write them separately in character variables and then print them out using theadjustl() function in fortran:
program csv
implicit none
integer, parameter :: dp = kind(1.0d0)
integer, parameter :: nn = 3
real(dp), parameter :: floatarray(nn) = [ -1.0_dp, -2.0_dp, -3.0_dp ]
integer :: ii
character(30) :: buffer(nn+2), myformat
! Create format string with appropriate number of fields.
write(myformat, "(A,I0,A)") "(A,", nn + 2, "(',',A))"
! You should execute the following lines in a loop for every line you want to output
write(buffer(1), "(F20.2)") 1.0_dp ! a_float
write(buffer(2), "(F20.2)") 2.0_dp ! a_different_float
do ii = 1, nn
write(buffer(2+ii), "(F20.3)") floatarray(ii)
end do
write(*, myformat) "a_string", (trim(adjustl(buffer(ii))), ii = 1, nn + 2)
end program csv
The demonstration above is only for one output line, but you can easily write a loop around the appropriate block to execute it for all your output lines. Also, you can choose different numerical format for the different entries, if you wish.
What is the best way to generate a name for some temporary context which is guaranteed to be unique (context with this name must not exist in the system)?
The following expression will generate a context name that is guaranteed not to conflict with any loaded context:
First#Contexts[] //.
c_ /; MemberQ[Contexts[], c] :>
"Context"~~ToString[RandomInteger[1000000]]~~"`"
It makes no attempt to account for contexts that are not yet loaded. As written, this expression could be used up to 1,000,000 times before running out of names. Adjust the fixed string ("Context") and name count (1000000) to suit your taste.
Update
As #Leonid points out in a comment, empty contexts will not be listed in Contexts[]. Therefore, it is strictly speaking possible that this expression could return the name of an existing empty context.
UUIDs
For all practical purposes, generating a name from a number randomly selected from a large enough range would work, e.g.
"Context"~~ToString[RandomInteger[2^128]]~~"`"
In a similar vein, one could use a UUID. UUIDs are routinely used as identifiers that are phenomenally likely to be unique across all network nodes as well:
Needs["JLink`"]
LoadJavaClass["java.util.UUID"]
"Context"~~
StringReplace[JavaBlock#java`util`UUID`randomUUID[]#toString[], "-" -> ""]~~
"`"
I can suggest a function I used here:
Clear[unique];
unique[sym_] :=
ToExpression[
ToString[Unique[sym]] <>
StringReplace[StringJoin[ToString /# Date[]], "." :> ""]];
You can replace the ToExpression by StringJoin[...,"`"] to tailor it to your needs.
Another option would be to look at all starting contexts (before the first backquote), find their string length and then generate a string (maybe random, but that isn't necessary) that is at least one character longer than the others. This is guaranteed to be unique, and there isn't even a theoretical possibility of a collision as in some of the other solutions.
sl = (StringSplit[#, "`"][[1]] & /# Contexts[] // StringLength // Max )
Out[349]= 30
In[353]:= "c" ~~ ToString[10^sl] ~~ "`"
Out[353]= "c1000000000000000000000000000000`"
A disadvantage of this method would be that the context names get longer after each repeated application of this method. ;-) If that's a problem we could create a unique name based on the set of longest context names using Cantor's diagonal procedure.
Is Unique perhaps what you're looking for?
This is really an example illustrating Alexey's response to Sjoerd's answer/question. From a fresh kernel on my machine, the following code
Begin["myContext3`"];
Unique["myContext"]
Yields "myContext3". Thus, clearly, Unique (the first thing I thought of) does not work.
Incidentally, I would have just added a comment to Sjoerd's response, but I don't know how to include the accent symbol used to denote a context inline. Does anyone here know how to do this?