Identify this scripting language/format - windows

This script is part of a legacy code that runs to create a GUI interface for editing text files to be used in analysis codes. There is a windows command script that references a ".sed" file which controls the formatting, editing, and help menu for the GUI. I would like to identify the coding language/rules used in these ".sed" files so that I can make a new more complicated input text file with descriptions of inputs.
Ideally, I would like to be able to use this code to create/edit ".csv" files which can be edited in Excel. This would potentially mean needing to avoid the set variable sizes/padding in the #file block of code below.
Any googling attempts to find more about the coding result in unix sed instructions that are not helpful.
UPDATE: I did find an additional exe in the shell folder of the legacy code for "sedwin.exe". When googled this seems to refer to an old "SEDT text editor for MS-DOS".
An example section from a ".sed" file is below:
<code>
#rem( version description information here )
#version(){"2.0"}
%--------------------------------------------------------------------
#file(seal2,native){
title1(A80);
title2(A80);
title3(A80);
r(G10),del(A1),ll(G10),c(G10),lg(G10),dg(G10),ngroov(I10);
ncase(I10),necc(I10),necase(I10),#for(i,1,necase,0){entlos[i](G10)};
#for(i,1,ncase,0){
speed[i](G10),ro[i](G10),nu[i](G10),delp[i](G10);
}
}
#edit(seal2){
#prompted(22,5,"SEAL2 Input Data"){
#help(){
" Code Name Here
"",
"Code description here"
};
#icon("Titles",titles){#titles();}
#icon("Seal Parameters",seal){#seal();}
#icon("Speed, Fluid Parameters",cases){#cases();}
}
}
#titles(){
#prompted(1,7,"Three Title Lines"){
#help(){
"These title lines will appear on the output of",
"the program.",
"",
"They are useful for identifying the output but",
"do not directly affect the results."
};
#datum("",title1,75,"");
#datum("",title2,75,"");
#datum("",title3,75,"");
}
}
#seal(){
#prompted(12,8,"Seal Parameters"){
#help(){
"Descriptions of inputs in this window.",
};
#datum("Shaft Radius (in)",r,15,"0.");
#float_check("Must be > 0.0","(%g)>0.");
#datum("Land Length (in)",ll,15,"0.");
#float_check("Must be > 0.0","(%g)>0.");
#datum("Seal Radial Clearance (in)",c,15,"0.");
#float_check("Must be > 0.0","(%g)>0.");
#datum("Groove Length (in)",lg,15,"0.");
#float_check("Must be >= 0.0","(%g)>=0.");
#datum("Groove Depth (in)",dg,15,"0.");
#float_check("Must be >= 0.0","(%g)>=0.");
#datum("Number of Grooves (0=plain seal)",ngroov,15,"0");
#int_check("Must be >= 0","(%d)>=0");
#datum("Number of Eccentricities",necc,15,"1");
#int_check("Must be between 1 and 10","(%d)>0&&(%d)<11");
#icon("Entrance Loss Cases",losses){#losses();}
}
}
#new_file(seal2){
file_type=seal2;
unit_type=native;
titles=New;
title1=;
title2=;
title3="(SEAL2 Data File)";
del=",";
seal=New;
ll=0.0;
r=0.0;
c=0.0;
lg=0.0;
dg=0.0;
ngroov=0;
losses=New;
necc=1;
necase=1;
entlos[1...necase]=0.1;
cases=New;
ncase=1;
speed[1...ncase]=0.0;
delp[1...ncase]=0.0;
nu[1...ncase]=0.0;
ro[1...ncase]=0.0;
}
</code>

Related

Google Cloud DLP - CSV inspection

I'm trying to inspect a CSV file and there are no findings being returned (I'm using the EMAIL_ADDRESS info type and the addresses I'm using are coming up with positive hits here: https://cloud.google.com/dlp/demo/#!/). I'm sending the CSV file into inspect_content with a byte_item as follows:
byte_item: {
type: :CSV,
data: File.open('/xxxxx/dlptest.csv', 'r').read
}
In looking at the supported file types, it looks like CSV/TSV files are inspected via Structured Parsing.
For CSV/TSV does that mean one can't just sent in the file, and needs to use the table attribute instead of byte_item as per https://cloud.google.com/dlp/docs/inspecting-structured-text?
What about for XSLX files for example? They're an unspecified file type so I tried with a configuration like so, but it still returned no findings:
byte_item: {
type: :BYTES_TYPE_UNSPECIFIED,
data: File.open('/xxxxx/dlptest.xlsx', 'rb').read
}
I'm able to do inspection and redaction with images and text fine, but having a bit of a problem with other file types. Any ideas/suggestions welcome! Thanks!
Edit: The contents of the CSV in question:
$ cat ~/Downloads/dlptest.csv
dylans#gmail.com,anotehu,steve#example.com
blah blah,anoteuh,
aonteuh,
$ file ~/Downloads/dlptest.csv
~/Downloads/dlptest.csv: ASCII text, with CRLF line terminators
The full request:
parent = "projects/xxxxxxxx/global"
inspect_config = {
info_types: [{name: "EMAIL_ADDRESS"}],
min_likelihood: :POSSIBLE,
limits: { max_findings_per_request: 0 },
include_quote: true
}
request = {
parent: parent,
inspect_config: inspect_config,
item: {
byte_item: {
type: :CSV,
data: File.open('/xxxxx/dlptest.csv', 'r').read
}
}
}
dlp = Google::Cloud::Dlp.dlp_service
response = dlp.inspect_content(request)
The CSV file I was testing with was something I created using Google Sheets and exported as a CSV, however, the file showed locally as a "text/plain; charset=us-ascii". I downloaded a CSV off the internet and it had a mime of "text/csv; charset=utf-8". This is the one that worked. So it looks like my issue was specifically due the file being an incorrect mime type.
xlsx is not yet supported. Coming soon. (Maybe that part of the question should be split out from the CSV debugging issue.)

How to combine lines from separate text files into one new text tile using Shell Script

I just want to preface this by saying I am an absolute noob to shell script, so stick with me here.
Essentially, I am trying to combine lines of text from separate test files (one line from HC_01_MNoFraming.txt added at the end of the corresponding line from HC_01_Gist.txt) into a single line of text on a newly created text file called HC_01_MGNoFraming.txt. This combination of the lines of text would occur for each of the four lines of text in each text file. Below is an example:
HC_01_Gist.txt
12.0130754383 33.0026698754 117.01282238700001 182.01823507400002 201.005570202 220.010026843 352.023495725 478.012859369 518.0072172580001
12.012680624100001 56.0118834624 144.026174161 167.018345335 228.002317522 247.00666698400002 356.027611312 434.014129075 478.013307259
56.01133045 142.00709709999998 207.0121417 331.01635039999996 350.0040858 369.0084907 390.01512310000004 409.0028586 430.0265601 512.0175128999999 556.0159299
12.012615221199999 176.01120546400003 199.00347731 243.019245908 430.00998814900004 472.02324678400004 495.01541966900004
HC_01_MNoFraming
262.006565969 283.013202477 375.015624781 541.016055549
33.019710919299996 497.017719727 537.0287543219999
121.0167582 226.0165682
75.01583263170001 113.024949998 262.023691934 386.010783941 537.011983108
HC_01_MGNoFraming.txt
12.0130754383 33.0026698754 117.01282238700001 182.01823507400002 201.005570202 220.010026843 352.023495725 478.012859369 518.0072172580001 262.006565969 283.013202477 375.015624781 541.016055549
12.012680624100001 56.0118834624 144.026174161 167.018345335 228.002317522 247.00666698400002 356.027611312 434.014129075 478.013307259 33.019710919299996 497.017719727 537.0287543219999
56.01133045 142.00709709999998 207.0121417 331.01635039999996 350.0040858 369.0084907 390.01512310000004 409.0028586 430.0265601 512.0175128999999 556.0159299 121.0167582 226.0165682
12.012615221199999 176.01120546400003 199.00347731 243.019245908 430.00998814900004 472.02324678400004 495.01541966900004 75.01583263170001 113.024949998 262.023691934 386.010783941 537.011983108
I tried several things but the simplest solution I could find online was just using the paste command like this:
#(A) First Timing File = Timing File you Want to Be in Front
A=HC_01_Gist.txt
#(B) Second Timing File Timing File you Want to Be in Back
B=HC_01_MNoFraming.txt
#(C) Output Timing File Name C=HC_01_MGNoFraming.txt
#------------------------------------------------------------
paste $B $A
paste $B $A >$C
Looking at the terminal output, this is what this code gives:
262.006512.0130754383 33.0026698754 117.01282238700001 182.01823507400002 201.005570202 220.010026843 352.023495725 478.012859369 518.0072172580001
33.0197112.012680624100001 56.0118834624 144.026174161 167.018345335 228.002317522 247.00666698400002 356.027611312 434.014129075 478.013307259
121.016756.01133045 142.00709709999998 207.0121417 331.01635039999996 350.0040858 369.0084907 390.01512310000004 409.0028586 430.0265601 512.0175128999999 556.0159299
75.0158312.012615221199999 176.01120546400003 199.00347731 243.019245908 430.00998814900004 472.02324678400004 495.01541966900004
It almost looks like in terminal that the paste command starts working but then after the first data point of HC_01_MNoFraming, the paste jumbles up the text and then resumes with the second data point of HC_01_Gist.txt.
Looking at the HC_01_MGNoFraming.txt file that the code creates:
262.006565969 283.013202477 375.015624781 541.016055549
12.0130754383 33.0026698754 117.01282238700001 182.01823507400002 201.005570202 220.010026843 352.023495725 478.012859369 518.0072172580001
33.019710919299996 497.017719727 537.0287543219999
12.012680624100001 56.0118834624 144.026174161 167.018345335 228.002317522 247.00666698400002 356.027611312 434.014129075 478.013307259
121.0167582 226.0165682
56.01133045 142.00709709999998 207.0121417 331.01635039999996 350.0040858 369.0084907 390.01512310000004 409.0028586 430.0265601 512.0175128999999 556.0159299
75.01583263170001 113.024949998 262.023691934 386.010783941 537.011983108
12.012615221199999 176.01120546400003 199.00347731 243.019245908 430.00998814900004 472.02324678400004 495.01541966900004
I'm wondering about two things:
1)How can I fix the output .txt file to have the data points in-line 2) how can I ensure that the data points for HC_01_Gist.txt go before the data points for HC_01_MNoFraming.
Thanks and I'd be more than happy to clarify my problem and answer any questions you have.

Which programming/markup language is this?

I'm working on a game with a team and I'm new to their project.
They are using an own engine which is programmed by a separate guy which is the leader of the team. He sent me the game project and the engine. He is 2 weeks away and I should start to program basic player movement.
The scripting is in C# and Lua and that's okay for me but often I see weird "script" file in the project folder, next to the game code. I'm not sure what this is? It's not xml nor json. Does anybody know what language this is? I can't ask him because he is away.
This "code" was in a file called scenery_backstream_bstr.cac:
#CAC startdoc __lenver##__kestd%BASEVER?1.0
&GVAR __kestd%GENGRP{ #CAC(BCKSTR} </ BASE PARTITION\>
{
5e24f2b0c52ffc549a40583c2a44082e
425bc050294b7dd9ac428da5bcc8cffe
3c681b7f4513938488f0a61556f34f3e
82100e8b272a6ebf4cc052178fbf55b9
d0b7e6ee96f6fdf853ee9795d02e97fb
} __strideof(GENGRP{#CAC(BCKSTR)}} **: (0xff) ? 0xff = 0 {
__kestd_drawbucket
{
SHADERPARTITION_0x0##GEN_REFLCETION
{
UNIM##((0x) ?? 0xff $ HLS_D3DBCSTR) == $AS_BASEPOINTER ? § : ^; </ IF DRAWBUCKET IS UNLOADED GET THE BASE POINTER OR NULL (§)\>
{
144b1cd49af1
3c2ae146eb51
2b2cf74be189
7a31fa2061f9
980376b28884
b88694dac0ed
dbb36cc7c931
d7a6fb3b98cb
81559847891d
fbbad67448bb
1bf499e6f357
083a7f0c8ce5
610e6d34a2ab
f242419f11b7
fdbbd067898c
01eee4a57d91
46a1cf49c308
dc33149a6783
2c77813db5fa
a9dbe5b6d114
5407cf314ec7
d5c687ede619
15a2dcfa6e4d
96a551aa3b14
e48cf48c74a9
1258c03dca75
53bc36a37f2c
260639258063
dd4d19659f93
932865eebc57
917dd4af7c1c
f98d9849201f
542b441d95da
4d338c35cb23
094b2055f213
bae9ddf89d22
84476019ab95
7b6f036a1f58
759284affea7
01232cad190e
c3a63151b473
6f7fafee368c
f5f76ebe20bf
e18e59e4e259
e13840a2f414
f7db178cbbb6
d253d62c5221
beb0d2bafd03
32acb34255b0
08fed326a21b
0b0c9a03e719
b72f71bfb5d4
0e7f863f0596
050d7f3aaefb
84e28725c90d
c6b297e05d47
86102bf15a96
b0d3383844ad
0376ec0759bc
e921dd5728da
df0aa7a863f4
f20347d37745
91e1ee783163
23649ec06a1e
}
}
}
}
#CAC endscript AS##(;)- (OK(§)(){ 0xff}
Ok it has been answered there was a txt file explaining all file types.
Cac is the a engine format and this is a file which contains indices which material has which shader in a scene. A drawbucket is here called a shader with resources.
Alright so it wasn't a public language.
Thanks anyways :)

Reading and Writing File Contents From HTTP Post Data In Bash

Problem Statement:
I am trying to upload a file through an HTML form using an HTTP post request and then write it to a file called configuration.xml on my local server. I can only use the stock capabilities of the server, so, as much as I'd love to, I can't use cURL, PHP, Perl, or anything that I'd have to install on the server. What I have tried doing is having the HTML form open a CGI file as the form action and all this CGI file does is run the Bash script with the proper HTML formatting. I would run the Bash script directly from the HTML form, but my research led me to believe that this isn't possible without having to edit .htaccess or other hacky alternatives, which are not roads I want to go down. (If this can be done in a reasonable fashion, please enlighten me!) Regardless, I am able to successfully run the Bash script. I know this because I put a "touch configuration.xml" command in the script and it creates it every time. My script is also able to tell that it is an HTTP Post, as shown by the echoed text in the browser, but then I can't seem to be able to properly read any data from the file. I tried echoing the data as well as redirecting the read data to a file, but nothing appeared in the browser and nothing wrote to the file I specified. This very well may be me not knowing Bash scripting well enough or something silly like that, but I really don't know how to proceed from here.
Code:
UploadToServer.html:
<form action="run_script.cgi" method="POST" enctype="multipart/form-data">
<input type="file" name="file" />
<input type="submit" name="submit" value="Submit">
</form>
run_script.c:
Note: I compile this to a CGI file with the following command: gcc run_script.c -o run_script.cgi
#include <stdlib.h>
#include <stdio.h>
int main() {
system("./test.sh &");
printf("Content-Type: text/html\r\n\r\n");
printf(""); // print blank line for proper HTML header formatting
printf("<html>\n");
printf("</HTML>\n");
}
test.sh:
The non-commented code in the second if statement is from here. The commented code is from here.
#!/bin/bash
touch configuration.xml
if [[ $REQUEST_METHOD = 'POST' ]]; then
echo "this is a post!"
if [ "$CONTENT_LENGTH" -gt 0 ]; then
echo "entered second if statement!"
# read -n $CONTENT_LENGTH POST_DATA <&0
# echo "$CONTENT_LENGTH"
while read line
do eval "echo ${line}"
done
fi
fi
I also tried the approach in the third code block of this post, but didn't get any output. I also looked through this post, but it doesn't seem to grab all the data from the file like I need to. I also tried the approach of just using a CGI file like suggested in this post (_http://blog.purplepixie.org/2013/08/cc-cgi-file-upload/), but, once again, no output. I've been looking through the Apache error log as I try new things and no errors come up.
Anybody have any ideas on what I could possibly be doing wrong? Is there a different approach worth looking into? Any suggestions are greatly appreciated!
I figured out how to do it, with some help from my friends. I ended up doing it all in a CGI script and foregoing the Bash component. While this isn't what I asked for in my original question, it gets the job done for me, which is really what the question was asking.
The following is the C code I'm now using to successfully write the file on the server:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
void print_empty_html_page();
int main() {
char * req_method_str = getenv("REQUEST_METHOD");
if (req_method_str != NULL) {
if (strcmp(req_method_str, "POST") == 0) {
// process POST arguments
char * len_str = getenv("CONTENT_LENGTH");
if (len_str != NULL) {
int len = atoi(len_str);
if (len > 0) {
FILE * fp;
fp = fopen("file.xml", "w");
char * postdata = malloc((len + 1) * sizeof(char));
fread(postdata, sizeof(char), len, stdin);
postdata[len] = '\0';
fprintf(fp, "%s\n", postdata);
free(postdata);
fclose(fp);
}
system("sed -e '/Content/d' -e '/[-][-][*][*][*][*][*]/d' -e '/^[s]*$/d' -e '/WebKitFormBoundary/d' -e '/Submit/d' < file.xml > file_trimmed.xml");
system("rm file.xml");
}
}
}
print_empty_html_page();
return 0;
}
void print_empty_html_page() {
// Send the content type, letting the browser know this is HTML
printf("Content-type: text/html\r\n\r\n");
// Header information that prevents browser from caching
printf(
"<META HTTP-EQUIV=\"CACHE-CONTROL\" CONTENT=\"NO-CACHE, NO-STORE\">\r\n\r\n");
// Top of the page
printf("<html>\n");
printf("<BODY>\n");
// Finish up the page
printf("</BODY></html>\n");
}
Note: This method writes the entire HTTP POST to the file 'file.xml'. The system call to 'sed' is to remove the tags from the HTTP POST that don't correspond to the actual data in the file that was uploaded. If you need to check for additional unwanted lines, just add another -e '/<line_with_expression_to_delete>/d' in that sed call, where line_with_expression_to_delete is the expression you want to match and then delete all lines containing that expression. I couldn't figure out how to delete all the blank lines in the newly uploaded file, even though '/^[s]*$/d' should do that, according to my research. Gonna have to look into that more...
Also note: This method only works for uploading text files. It does not work for other file types, such as JPEGs or OGGs.
Hopefully this helps some other people with the same problem. Let me know if you have any questions.

Forcing a package's function to use user-provided function

I'm running into a problem with the MNP package which I've traced to an unfortunate call to deparse (whose maximum width is limited to 500 characters).
Background (easily skippable if you're bored)
Because mnp uses a somewhat idiosyncratic syntax to allow for varying choice sets (you include cbind(choiceA,choiceB,...) in the formula definition), the left hand side of my formula call is 1700 characters or so when model.matrix.default calls deparse on it. Since deparse supports a maximum width.cutoff of 500 characters, the sapply(attr(t, "variables"), deparse, width.cutoff = 500)[-1L] line in model.matrix.default has as its first element:
[1] "cbind(plan1, plan2, plan3, plan4, plan5, plan6, plan7, plan8, plan9, plan10, plan11, plan12, plan13, plan14, plan15, plan16, plan17, plan18, plan19, plan20, plan21, plan22, plan23, plan24, plan25, plan26, plan27, plan28, plan29, plan30, plan31, plan32, plan33, plan34, plan35, plan36, plan37, plan38, plan39, plan40, plan41, plan42, plan43, plan44, plan45, plan46, plan47, plan48, plan49, plan50, plan51, plan52, plan53, plan54, plan55, plan56, plan57, plan58, plan59, plan60, plan61, plan62, plan63, "
[2] " plan64, plan65, plan66, plan67, plan68, plan69, plan70, plan71, plan72, plan73, plan74, plan75, plan76, plan77, plan78, plan79, plan80, plan81, plan82, plan83, plan84, plan85, plan86, plan87, plan88, plan89, plan90, plan91, plan92, plan93, plan94, plan95, plan96, plan97, plan98, plan99, plan100, plan101, plan102, plan103, plan104, plan105, plan106, plan107, plan108, plan109, plan110, plan111, plan112, plan113, plan114, plan115, plan116, plan117, plan118, plan119, plan120, plan121, plan122, plan123, "
[3] " plan124, plan125, plan126, plan127, plan128, plan129, plan130, plan131, plan132, plan133, plan134, plan135, plan136, plan137, plan138, plan139, plan140, plan141, plan142, plan143, plan144, plan145, plan146, plan147, plan148, plan149, plan150, plan151, plan152, plan153, plan154, plan155, plan156, plan157, plan158, plan159, plan160, plan161, plan162, plan163, plan164, plan165, plan166, plan167, plan168, plan169, plan170, plan171, plan172, plan173, plan174, plan175, plan176, plan177, plan178, plan179, "
[4] " plan180, plan181, plan182, plan183, plan184, plan185, plan186, plan187, plan188, plan189, plan190, plan191, plan192, plan193, plan194, plan195, plan196, plan197, plan198, plan199, plan200, plan201, plan202, plan203, plan204, plan205, plan206, plan207, plan208, plan209, plan210, plan211, plan212, plan213, plan214, plan215, plan216, plan217, plan218, plan219, plan220, plan221, plan222, plan223, plan224, plan225, plan226, plan227, plan228, plan229, plan230, plan231, plan232, plan233, plan234, plan235, "
[5] " plan236, plan237, plan238, plan239, plan240, plan241, plan242, plan243, plan244, plan245, plan246, plan247, plan248, plan249, plan250, plan251, plan252, plan253, plan254, plan255, plan256, plan257, plan258, plan259, plan260, plan261, plan262, plan263, plan264, plan265, plan266, plan267, plan268, plan269, plan270, plan271, plan272, plan273, plan274, plan275, plan276, plan277, plan278, plan279, plan280, plan281, plan282, plan283, plan284, plan285, plan286, plan287, plan288, plan289, plan290, plan291, "
[6] " plan292, plan293, plan294, plan295, plan296, plan297, plan298, plan299, plan300, plan301, plan302, plan303, plan304, plan305, plan306, plan307, plan308, plan309, plan310, plan311, plan312, plan313)"
When model.matrix.default tests this against the variables in the data.frame, it returns an error.
The problem
To get around this, I've written a new deparse function:
deparse <- function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
}
However, when I run mnp again and step through, it returns the same error for the same reason (base::deparse is being run, not my deparse).
This is somewhat surprising to me, as what I expect is more typified by this example, where the user-defined function temporarily over-writes the base function:
> print <- function() {
+ cat("user-defined print ran\n")
+ }
> print()
user-defined print ran
I realize the right way to solve this problem is to rewrite model.matrix.default, but as a tool for debugging I'm curious how to force it to use my deparse and why the anticipated (by me) behavior is not happening here.
The functions fixInNamespace and assignInNamespace are provided to allow editing of existing functions. You could try ... but I will not since mucking with deparse looks too dangerous:
assignInNamespace("deparse",
function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
} , "base")
There is an indication on the help page that the use of such functions has restrictions and I would not be surprised that such core function might have additional layers of protection. Since it works via side-effect, you should not need to assign the result.
This is how packages with namespaces search for functions, as described in Section 1.6, Package Namespaces of Writing R Extensions
Namespaces are sealed once they are loaded. Sealing means that imports
and exports cannot be changed and that internal variable bindings
cannot be changed. Sealing allows a simpler implementation strategy
for the namespace mechanism. Sealing also allows code analysis and
compilation tools to accurately identify the definition corresponding
to a global variable reference in a function body.
The namespace controls the search strategy for variables used by
functions in the package. If not found locally, R searches the package
namespace first, then the imports, then the base namespace and then
the normal search path.

Resources