Unable to increment last 2 digit of variable declared in file using script - shell

I have the file given below:
elix554bx.xayybol.42> vi setup.REVISION
# Revision information
setenv RSTATE R24C01
setenv CREVISION X3
exit
My requirement is to read RSTATE from file and then increment last 2 digits of RSTATE in setup.REVISION file and overwrite into same file.
Can you please suggest how to do this?

If you're using vim, then you can use the sequence:
/RSTATE/
$<C-a>:x
The first line is followed by a return and searches for RSTATE. The second line jumps to the end of the line and uses Control-a (shown as <C-a> above, and in the vim documentation) to increment the number. Repeat as often as you want to increment the number. The :x is also followed by a return and saves the file.
The only tricky bit is that the leading 0 on the number makes vim think the number is in octal, not decimal. You can override that by using :set nrformats= followed by return to turn off octal and hex; the default value is nrformats=octal,hex.
You can learn an awful lot about vim from the book Practical Vim: Edit Text at the Speed of Thought by Drew Neil. This information comes from Tip 10 in chapter 2.

Here's an awk one-liner type solution:
awk '{
if ( $0 ~ 'RSTATE' ) {
match($0, "[0-9]+$" );
sub( "[0-9]+$",
sprintf( "%0"RLENGTH"d", substr($0, RSTART, RSTART+RLENGTH)+1 ),
$0 );
print; next;
} else { print };
}' setup.REVISION > tmp$$
mv tmp$$ setup.REVISION
Returns:
setenv RSTATE R24C02
setenv CREVISION X3
exit
This will handle transitions from two to three to more digits appropriately.

I wrote for you a class.
class Reader
{
public string ReadRs(string fileWithPath)
{
string keyword = "RSTATE";
string rs = "";
if(File.Exists(fileWithPath))
{
StreamReader reader = File.OpenText(fileWithPath);
try
{
string line = "";
bool finded = false;
while (reader != null && !finded)
{
line = reader.ReadLine();
if (line.Contains(keyword))
{
finded = true;
}
}
int index = line.IndexOf(keyword);
rs = line.Substring(index + keyword.Length +1, line.Length - 1 - (index + keyword.Length));
}
catch (IOException)
{
//Error
}
finally
{
reader.Close();
}
}
return rs;
}
public int GetLastTwoDigits(string rsState)
{
int digits = -1;
try
{
int length = rsState.Length;
//Get the last two digits of the rsstate
digits = Int32.Parse(rsState.Substring(length - 2, 2));
}
catch (FormatException)
{
//Format Error
digits = -1;
}
return digits;
}
}
You can use this as exists
Reader reader = new Reader();
string rsstate = reader.ReadRs("C://test.txt");
int digits = reader.GetLastTwoDigits(rsstate);

Related

Expert Advisor timefilter doesn't work (mql5)?

I can't figure out why my timefilter doesn't work. Let's say I would like to only enter to positions between 7:35-11:30 and 14:30-22:30 and I don't want to enter a position on Friday.
The time filter only works when I create a simple EA with only a trade.Buy function and no other conditions.
The more complex EA should only enter a position when the vaule of the Supertrend indicator becomes higher/lower than the price and only in the given time intervals.
It should close the position at the next sell/buy signal (if it was a buy position then the position should be closed at the next 'sell' signal' ). When closing positions the time interval shouldn't matter it should only mater when entering a new position.
The 'TradingIsAllowed' variable should be 'true' when the current time is in the allowed time intervals but it always returns false for some reason and I can't figure out why.
It works perfectly fine when I don't use the supertrend and close trades with a simple tp/sl.
Could you please help me?
#include <Trade\Trade.mqh>
CTrade trade;
ulong posTicket;
input double Lots=0.1;
int stHandle;
int totalBars;
input ENUM_TIMEFRAMES Timeframe = PERIOD_CURRENT;
input int Periods =12;
input double Multiplier = 3.0;
//for the timefilter
input string StartTradingTime="07:35";
input string StopTradingTime="11:30";
input string StartTradingTime2="14:35";
input string StopTradingTime2="22:30";
string CurrentTime;
bool TradingIsAllowed=false;
bool TradingIsAllowed2=false;
int OnInit(){
totalBars=iBars(_Symbol,Timeframe);
stHandle = iCustom(_Symbol, Timeframe, "Supertrend.ex5", Periods, Multiplier);
return(INIT_SUCCEEDED);
}
void OnDeinit(const int reason){
}
void OnTick(){
//for the timefilter
datetime LocalTime=TimeLocal();
string HoursAndMinutes=TimeToString(LocalTime,TIME_MINUTES);
string YearAndDate=TimeToString(LocalTime, TIME_DATE);
MqlDateTime DateTimeStructure;
TimeToStruct(LocalTime, DateTimeStructure);
int DayOfWeek=DateTimeStructure.day_of_week;
datetime time = TimeLocal();
CurrentTime=TimeToString(time,TIME_MINUTES);
//this should only run if there is a new bar
int bars=iBars(_Symbol, Timeframe);
if(totalBars !=bars){
totalBars=bars;
double st[];
CopyBuffer(stHandle,0,0,3,st);
double close1 = iClose(_Symbol, Timeframe, 1);
double close2 = iClose(_Symbol, Timeframe, 2);
//BUY CONDITION
if(close1 > st[1] && close2 < st[0]){
if(posTicket > 0 ){
if(PositionSelectByTicket(posTicket)){
if(PositionGetInteger(POSITION_TYPE) == POSITION_TYPE_SELL){
if (trade.PositionClose(posTicket)){
Print(__FUNCTION__," > Pos ", posTicket, "was closed..");
}
}
}
}
if(CheckTradingTime()==true || CheckTradingTime2()==true){
if(PositionsTotal()==0 && DayOfWeek!=5){
Print(__FUNCTION__, " > BOUGHT");
if(trade.Buy(Lots, _Symbol)){
if(trade.ResultRetcode() == TRADE_RETCODE_DONE){
posTicket= trade.ResultOrder();
}
}
}
}
}
else if(close1 < st[1] && close2 > st[0]){
if(posTicket > 0 ){
if(PositionSelectByTicket(posTicket)){
if(PositionGetInteger(POSITION_TYPE) == POSITION_TYPE_BUY){
if (trade.PositionClose(posTicket)) {
Print(__FUNCTION__," > Pos ", posTicket, "was closed..");
}
}
}
}
if(CheckTradingTime()==true || CheckTradingTime2()==true){
if(PositionsTotal()==0 && DayOfWeek!=5){
Print(__FUNCTION__, " > SOLD");
if(trade.Sell(Lots, _Symbol)){
if(trade.ResultRetcode() == TRADE_RETCODE_DONE){
posTicket= trade.ResultOrder();
}
}
}
}
}
}
Comment (
"TradingIsAllowed", TradingIsAllowed, TradingIsAllowed2, "\n", //TradingIsAllowed always returns false..
"Current Time=", CurrentTime,"\n",
"Trading Session1=", StartTradingTime,"-" ,StopTradingTime, "\n",
"Trading Session2=", StartTradingTime2, "-", StopTradingTime2,"\n",
"Day of Week", DayOfWeek
);
}
//trading session 1
bool CheckTradingTime()
{
if(StringSubstr(CurrentTime,0,5)==StartTradingTime)
TradingIsAllowed=true;
if(StringSubstr(CurrentTime,0,5)==StopTradingTime)
TradingIsAllowed=false;
return TradingIsAllowed;
}
//trading session 2
bool CheckTradingTime2()
{
if(StringSubstr(CurrentTime,0,5)==StartTradingTime2)
TradingIsAllowed2=true;
if(StringSubstr(CurrentTime,0,5)==StopTradingTime2)
TradingIsAllowed2=false;
return TradingIsAllowed2;
}
you dont neet to write "(StringSubstr(CurrentTime,0,5)==StartTradingTime)"
you can write only "CurrentTime == StartTradingTime"

Using awk how do i pull out matched strings and other data in one command

I'm trying to parse over a number of files in a path for a certain string patterns (for e.g. new File()), which can occur over multiple lines in that file.
The information I'm trying to return is;
1 Filename/path
2 Number of occurrences of string pattern in the
file
3 Code found i.e new File()
4 Line number code found on
Here is an example file contents of test.txt;
new
File()
new File()
new
File()
Fil[![1]][1]e() new
new File() test new File()
Here is a picture of the file in notepad++
A more realistic real world example would be (made up code, not compilable);
package gw.plugin.document.impl
#Export
abstract class BaseLocalDocumentContentSource implements
InitializablePlugin
{
private static var DOCUMENTS_PATH = "documents.path"
public property get DemoDocumentsURL() : URL {
return new URL("file", "", DemoDocumentsPath)
}
construct() {
}
protected function buildDocumentsPath(documentRootDir : String,
documentTmpDir : String) {
if (DocumentsPathParameter.HasContent) {
DemoDocumentsPath = getAbsolutePath(DocumentsPathParameter,
documentRootDir)
if (!new test
File(DemoDocumentsPath).equals(new File(DocumentsPathParameter))) {
Logger.DOCUMENT.warn((typeof this).RelativeName + " has a
relative path specified for its documents.path parameter, so it will store
documents in the app container's temporary directory. For production use,
the configuration should be changed to a full directory path, not a
relative path")
DocumentsPath = getAbsolutePath(DocumentsPathParameter, documentTmpDir)
var file = new File(DocumentsPath)
if (!file.exists() && file.isDirectory()) {
file.mkdirs()
}
} else {
DocumentsPath = DemoDocumentsPath
}
}
Logger.DOCUMENT.info("Documents path: " + DocumentsPath)
}
protected function updateDocument(strDocUID : String, isDocument : InputStream) {
try {
var file = getDocumentFile(strDocUID)
if (!FileUtil.isFile( file ) || file.isReservedFileName()) {
throw new IllegalArgumentException("Document ${strDocUID} does not exist!")
}
var backupFile = new File(file.getPath() + ".bak")
if (not file.renameTo(backupFile) ) { // renamed physical file, 'file' still has previous name
throw new RuntimeException("Failed to rename file to ${backupFile}")
}
copyToFile(isDocument, file)
try {
backupFile.delete()
}
catch (e : Throwable) {
Logger.DOCUMENT.warn("DocMgmt failed to delete '${backupFile}'")
}
} catch (e : Exception) {
throw new RuntimeException("Exception encountered trying to update document with doc UID: ${strDocUID}", e)
}
}
protected function getDocumentFile(relativePath : String, checkDemoFolder : boolean) : File {
var file = new File(getDocumentsDir(), relativePath)
if (!file.exists() && checkDemoFolder) {
file = new File(getDemoDocumentsDir(), relativePath)
}
return file
}
protected function makeSubDirPath(diw : IDocumentInfoWrapper) : String {
var subDirPath = diw.getSubDirForDocument()
var dirDoc = new File(getDocumentsDir() + subDirPath)
if (not dirDoc.Directory) {
dirDoc.mkdirs()
}
return subDirPath
}
private static function getAbsolutePath(path : String, rootPath : String) : String {
var retVal = path
if (path.startsWith("\\") || path.startsWith("/") || (path.length() > 1 && path.charAt(1) == ":" as char)) {
retVal = path
} else {
retVal = rootPath + File.separator + path
}
try {
retVal = (new File(retVal)).getCanonicalPath()
} catch (e : IOException) {
throw new RuntimeException("Could not get absolute path from relative path: ${path}", e)
}
return retVal.replaceAll("\\\\","/")
}
}
I have looked at grep, pcregrep, sed and awk. The folder I'm searching is very large so I'm trying to return all data required in one command instead of running four commands and having to traverse the folder more than once.
I've found awk the most applicable but have very limited experience in all of the programs I mentioned and I'm I don't have authorisation to install pcregrep in the env so can't use that.
Here's my attempt for awk so far, it is wrong and probably poorly done so be gentle :)
awk '{
if(/new[[:space:]]*/) {
line1=NR;
code1=$0;
} if(/File\(\)/) {
count[$0]++;
line2=NR;
if(line1 != line2) {
code2=$0;
printf "Found on lines %d, %d, code = %s %s \nNumber of occurrences = %d", line1, line2, code1, code2, count[$0]
} else {
printf "Found on line %d, code = %s \nNumber of occurrences = %d", line1, code1, count[$0]
}
}
}' test.txt
I know that my count of occurences is incorrect as I'm counting the occurrences per match as opposed to the total in the file. I'm getting some weird output such as the below;
File()n lines 1, 2, code = new
Number of occurrences = 1
ound on line 3, code = new File()
Number of occurrences = 1
File()n lines 4, 8, code = new
Number of occurrences = 2
ound on line 9, code = File() new
Number of occurrences = 1
Where code2 is overwriting the first few words of the print statement and not printed where I'd expect.
Expected output would be something like;
test.txt (Filename)
5 (number of occurrences of new File() pattern)
new File() Found on lines 1 & 2
new File() Found on line 3
new File() Found on lines 4 & 9
new File() Found on line 10
new File() Found on line 10
Or something similar to this
Output of cat -vte test.txt is;
new^M$
File()^M$
new File()^M$
new ^M$
^M$
^M$
^M$
File()^M$
File() new^M$
new File() test new File()
Any help would be appreciated.
You may use this awk:
awk -v msg='new File() Found on line ' 'BEGIN {print ARGV[1], "(Filename)"} {while(match($0, /new[[:blank:]]+File\(\)/)) {print msg NR; ++n; $0 = substr($0, RSTART+RLENGTH)}} /new[[:blank:]]*$/ {p = NR; next} p && NF {if (/^[[:blank:]]*File\(\)/) {print msg p, "&", NR; ++n} p = 0} END {print n, "(number of occurrences of new File() pattern)"}' test.txt
test.txt (Filename)
new File() Found on line 1 & 2
new File() Found on line 3
new File() Found on line 4 & 8
new File() Found on line 10
new File() Found on line 10
5 (number of occurrences of new File() pattern)
A more readable form:
awk -v msg='new File() Found on line ' '
BEGIN {print ARGV[1], "(Filename)"}
{
while(match($0, /new[[:blank:]]+File\(\)/)) {
print msg NR
++n
$0 = substr($0, RSTART+RLENGTH)
}
}
/new[[:blank:]]*$/ {
p = NR
next
}
p && NF {
if (/^[[:blank:]]*File\(\)/) {
print msg p, "&", NR
++n
}
p = 0
}
END {
print n, "(number of occurrences of new File() pattern)"
}' test.txt
Taking code from #anubhava sir's post and editing it to work for multiple files edited and tested in GNU awk. It needs GNU awk since I am using ENDFILE option of it here, which will be executed at the completion of each file.
Added -v IGNORECASE="1" option since OP confirmed in comments that ignorecase is needed to match values.
awk -v IGNORECASE="1" -v msg='new File() Found on line ' '
BEGIN {print ARGV[1], "(Filename)"}
FNR==1 { p=n=0 }
{
while(match($0, /new[[:blank:]]+File\(\)/)) {
print msg FNR
++n
$0 = substr($0, RSTART+RLENGTH)
}
}
/new[[:blank:]]*$/ {
p = FNR
next
}
p && NF {
if (/^[[:blank:]]*File\(\)/) {
print msg p, "&", FNR
++n
}
p = 0
}
ENDFILE {
print n, "(number of occurrences of new File() pattern)"
}' *.txt

How I make a list of missing integer from a sequence using bash

I have a file let's say files_190911.csv whose contents are as follows.
EDR_MPU023_09_20190911080534.csv.gz
EDR_MPU023_10_20190911081301.csv.gz
EDR_MPU023_11_20190911083544.csv.gz
EDR_MPU023_14_20190911091405.csv.gz
EDR_MPU023_15_20190911105513.csv.gz
EDR_MPU023_16_20190911105911.csv.gz
EDR_MPU024_50_20190911235332.csv.gz
EDR_MPU024_51_20190911235400.csv.gz
EDR_MPU024_52_20190911235501.csv.gz
EDR_MPU024_54_20190911235805.csv.gz
EDR_MPU024_55_20190911235937.csv.gz
EDR_MPU025_24_20190911000050.csv.gz
EDR_MPU025_25_20190911000155.csv.gz
EDR_MPU025_26_20190911000302.csv.gz
EDR_MPU025_29_20190911000624.csv.gz
I want to make a list of missing sequence from those using bash script.
Every MPUXXX has its own sequence. So there are multiple series of sequences in that file.
The datetime for missing list will use from previous sequence.
From the sample above, the result will be like this.
EDR_MPU023_12_20190911083544.csv.gz
EDR_MPU023_13_20190911083544.csv.gz
EDR_MPU024_53_20190911235501.csv.gz
EDR_MPU025_27_20190911000302.csv.gz
EDR_MPU025_28_20190911000302.csv.gz
It would be simpler if there were only a single sequence.
So I can use something like this.
awk '{for(i=p+1; i<$1; i++) print i} {p=$1}'
But I know this can't be used for multiple sequence.
EDITED (Thanks #Cyrus!)
AWK is your friend:
#!/usr/bin/awk
BEGIN {
FS="[^0-9]*"
last_seq = 0;
next_serial = 0;
}
{
cur_seq = $2;
cur_serial = $3;
if (cur_seq != last_seq) {
last_seq = cur_seq;
ts = $4
prev = cur_serial;
} else {
if (cur_serial == next_serial) {
ts = $4;
} else {
for (i = next_serial; i < cur_serial; i++) {
print "EDR_MPU" last_seq "_" i "_" ts ".csv.gz"
}
}
}
next_serial = cur_serial + 1;
}
And then you do:
$ < files_190911.csv awk -f script.awk
EDR_MPU023_12_20190911083544.csv.gz
EDR_MPU023_13_20190911083544.csv.gz
EDR_MPU024_53_20190911235501.csv.gz
EDR_MPU025_27_20190911000302.csv.gz
EDR_MPU025_28_20190911000302.csv.gz
The assignment to FS= splits lines by the regex. The rest program detects holes in sequences and prints them with the appropriate timestamp.

Visual Studio code metrics misreporting lines of code

The code metrics analyser in Visual Studio, as well as the code metrics power tool, report the number of lines of code in the TestMethod method of the following code as 8.
At the most, I would expect it to report lines of code as 3.
[TestClass]
public class UnitTest1
{
private void Test(out string str)
{
str = null;
}
[TestMethod]
public void TestMethod()
{
var mock = new Mock<UnitTest1>();
string str;
mock.Verify(m => m.Test(out str));
}
}
Can anyone explain why this is the case?
Further info
After a little more digging I've found that removing the out parameter from the Test method and updating the test code causes LOC to be reported as 2, which I believe is correct. The addition of out causes the jump, so it's not because of braces or attributes.
Decompiling the DLL with dotPeek reveals a fair amount of additional code generated because of the out parameter which could be considered 8 LOC, but removing the parameter and decompiling also reveals generated code, which could be considered 5 LOC, so it's not simply a matter of VS counting compiler generated code (which I don't believe it should do anyway).
There are several common definitions of 'Lines Of Code' (LOC). Each tries to bring some sense to what I think of as an almost meaningless metric. For example google of effective lines of code (eLOC).
I think that VS is including the attribute as part of the method declaration and is trying to give eLOC by counting statements and even braces. One possiblity is that 'm => m.Test(out str)' is being counted as a statement.
Consider this:
if (a > 1 &&
b > 2)
{
var result;
result = GetAValue();
return result;
}
and this:
if (a> 1 && b >2)
return GetAValue();
One definition of LOC is to count the lines that have any code. This may even include braces. In such an extreme simplistic definition the count varies hugely on coding style.
eLOC tries to reduce or eliminate the influence of code style. For example, as may the case here, a declaration may be counted as a 'line'. Not justifying it, just explaining.
Consider this:
int varA = 0;
varA = GetAValue();
and this:
var varA = GetAValue();
Two lines or one?
It all comes down to what is the intent. If it is to measure how tall a monitor you need then perhaps use a simple LOC. If the intent is to measure complexity then perhaps counting code statements is better such as eLOC.
If you want to measure complexity then use a complexity metric like cyclomatic complexity. Don't worry about how VS is measuring LOC as, i think, it is a useless metric anyway.
With the tool NDepend we get a # Lines of Code (LoC) of 2 for TestMethod(). (Disclaimer I am one of the developers of this tool). I wrote an article about How do you count your number of Lines Of Code (LOC) ? that is shedding light on what is logical LoC, and how all .NET LoC counting tooling rely on the PDB sequence points technology.
My guess concerning this LoC value of 8 provided by VS metric, is that it includes the LoC of the method generated by the lambda expression + it includes the PDB sequences points related to open/ending braces (which NDepend doesn't). Also lot of gymnastic is done by the compiler to do what is called capturing the local variable str, but this shouldn't impact the #LoC that is inferred from the PDB sequence points.
Btw, I wrote 2 others related LoC articles:
Why is it useful to count the number of Lines Of Code (LOC) ?
Mythical man month : 10 lines per developer day
I was wondering about the Visual Studio line counting and why what I was seeing wasn't what was being reported. So I wrote a small C# console program to count pure lines of code and write the results to a CSV file (see below).
Open a new solution, copy and paste it into the Program.cs file, build the executable, and then you're ready to go. It's a .Net 3.5 application. Copy it into the topmost directory of your code base. Open a command window and run the executable. You get two prompts, first for name of the program/subsystem, and for any extra file types you want to analyze. It then writes the results to a CSV file in the current directory. Nice simple thing for your purposes or to hand to management.
Anyhoo, here it is, FWIW, and YMMV:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.IO;
namespace CodeMetricsConsole
{
class Program
{
// Concept here is that the program has a list of file extensions to do line counts on; it
// gets any extra extensions at startup from the user. Then it gets a list of files based on
// each extension in the current directory and all subdirectories. Then it walks through
// each file line by line and will display counts for that file and for that file extension.
// It writes that information to a CSV file in the current directory. It uses regular expressions
// on each line of each file to figure out what it's looking at, and how to count it (i.e. is it
// a line of code, a single or multi line comment, a multi-line string, or a whitespace line).
//
static void Main(string[] args)
{
try
{
Console.WriteLine(); // spacing
// prompt user for subsystem or application name
String userInput_subSystemName;
Console.Write("Enter the name of this application or subsystem (required): ");
userInput_subSystemName = Console.ReadLine();
if (userInput_subSystemName.Length == 0)
{
Console.WriteLine("Application or subsystem name required, exiting.");
return;
}
Console.WriteLine(); // spacing
// prompt user for additional types
String userInput_additionalFileTypes;
Console.WriteLine("Default extensions are asax, css, cs, js, aspx, ascx, master, txt, jsp, java, php, bas");
Console.WriteLine("Enter a comma-separated list of additional file extensions (if any) you wish to analyze");
Console.Write(" --> ");
userInput_additionalFileTypes = Console.ReadLine();
// tell user processing is starting
Console.WriteLine();
Console.WriteLine("Getting LOC counts...");
Console.WriteLine();
// the default file types to analyze - hashset to avoid duplicates if the user supplies extensions
HashSet allowedExtensions = new HashSet { "asax", "css", "cs", "js", "aspx", "ascx", "master", "txt", "jsp", "java", "php", "bas" };
// Add user-supplied types to allowedExtensions if any
String[] additionalFileTypes;
String[] separator = { "," };
if (userInput_additionalFileTypes.Length > 0)
{
// split string into array of additional file types
additionalFileTypes = userInput_additionalFileTypes.Split(separator, StringSplitOptions.RemoveEmptyEntries);
// walk through user-provided file types and append to default file types
foreach (String ext in additionalFileTypes)
{
try
{
allowedExtensions.Add(ext.Trim()); // remove spaces
}
catch (Exception e)
{
Console.WriteLine("Exception: " + e.Message);
}
}
}
// summary file to write to
String summaryFile = userInput_subSystemName + "_Summary.csv";
String path = Directory.GetCurrentDirectory();
String pathAndFile = path + Path.DirectorySeparatorChar + summaryFile;
// regexes for the different line possibilities
Regex oneLineComment = new Regex(#"^\s*//"); // match whitespace to two slashes
Regex startBlockComment = new Regex(#"^\s*/\*.*"); // match whitespace to /*
Regex whiteSpaceOnly = new Regex(#"^\s*$"); // match whitespace only
Regex code = new Regex(#"\S*"); // match anything but whitespace
Regex endBlockComment = new Regex(#".*\*/"); // match anything and */ - only used after block comment detected
Regex oneLineBlockComment = new Regex(#"^\s*/\*.*\*/.*"); // match whitespace to /* ... */
Regex multiLineStringStart = new Regex("^[^\"]*#\".*"); // match #" - don't match "#"
Regex multiLineStringEnd = new Regex("^.*\".*"); // match double quotes - only used after multi line string start detected
Regex oneLineMLString = new Regex("^.*#\".*\""); // match #"..."
Regex vbaComment = new Regex(#"^\s*'"); // match whitespace to single quote
// Uncomment these two lines to test your regex with the function testRegex() below
//new Program().testRegex(oneLineMLString);
//return;
FileStream fs = null;
String line = null;
int codeLineCount = 0;
int commentLineCount = 0;
int wsLineCount = 0;
int multiLineStringCount = 0;
int fileCodeLineCount = 0;
int fileCommentLineCount = 0;
int fileWsLineCount = 0;
int fileMultiLineStringCount = 0;
Boolean inBlockComment = false;
Boolean inMultiLineString = false;
try
{
// write to summary CSV file, overwrite if exists, don't append
using (StreamWriter outFile = new StreamWriter(pathAndFile, false))
{
// outFile header
outFile.WriteLine("filename, codeLineCount, commentLineCount, wsLineCount, mlsLineCount");
// walk through files with specified extensions
foreach (String allowed_extension in allowedExtensions)
{
String extension = "*." + allowed_extension;
// reset accumulating values for extension
codeLineCount = 0;
commentLineCount = 0;
wsLineCount = 0;
multiLineStringCount = 0;
// Get all files in current directory and subdirectories with specified extension
String[] fileList = Directory.GetFiles(Directory.GetCurrentDirectory(), extension, SearchOption.AllDirectories);
// walk through all files of this type
for (int i = 0; i < fileList.Length; i++)
{
// reset values for this file
fileCodeLineCount = 0;
fileCommentLineCount = 0;
fileWsLineCount = 0;
fileMultiLineStringCount = 0;
inBlockComment = false;
inMultiLineString = false;
try
{
// open file
fs = new FileStream(fileList[i], FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
using (TextReader tr = new StreamReader(fs))
{
// walk through lines in file
while ((line = tr.ReadLine()) != null)
{
if (inBlockComment)
{
if (whiteSpaceOnly.IsMatch(line))
{
fileWsLineCount++;
}
else
{
fileCommentLineCount++;
}
if (endBlockComment.IsMatch(line)) inBlockComment = false;
}
else if (inMultiLineString)
{
fileMultiLineStringCount++;
if (multiLineStringEnd.IsMatch(line)) inMultiLineString = false;
}
else
{
// not in a block comment or multi-line string
if (oneLineComment.IsMatch(line))
{
fileCommentLineCount++;
}
else if (oneLineBlockComment.IsMatch(line))
{
fileCommentLineCount++;
}
else if ((startBlockComment.IsMatch(line)) && (!(oneLineBlockComment.IsMatch(line))))
{
fileCommentLineCount++;
inBlockComment = true;
}
else if (whiteSpaceOnly.IsMatch(line))
{
fileWsLineCount++;
}
else if (oneLineMLString.IsMatch(line))
{
fileCodeLineCount++;
}
else if ((multiLineStringStart.IsMatch(line)) && (!(oneLineMLString.IsMatch(line))))
{
fileCodeLineCount++;
inMultiLineString = true;
}
else if ((vbaComment.IsMatch(line)) && (allowed_extension.Equals("txt") || allowed_extension.Equals("bas"))
{
fileCommentLineCount++;
}
else
{
// none of the above, thus it is a code line
fileCodeLineCount++;
}
}
} // while
outFile.WriteLine(fileList[i] + ", " + fileCodeLineCount + ", " + fileCommentLineCount + ", " + fileWsLineCount + ", " + fileMultiLineStringCount);
fs.Close();
fs = null;
} // using
}
finally
{
if (fs != null) fs.Dispose();
}
// update accumulating values
codeLineCount = codeLineCount + fileCodeLineCount;
commentLineCount = commentLineCount + fileCommentLineCount;
wsLineCount = wsLineCount + fileWsLineCount;
multiLineStringCount = multiLineStringCount + fileMultiLineStringCount;
} // for (specific file)
outFile.WriteLine("Summary for: " + extension + ", " + codeLineCount + ", " + commentLineCount + ", " + wsLineCount + ", " + multiLineStringCount);
} // foreach (all files with specified extension)
} // using summary file streamwriter
Console.WriteLine("Analysis complete, file is: " + pathAndFile);
} // try block
catch (Exception e)
{
Console.WriteLine("Error: " + e.Message);
}
}
catch (Exception e2)
{
Console.WriteLine("Error: " + e2.Message);
}
} // main
// local testing function for debugging purposes
private void testRegex(Regex rx)
{
String test = " asdfasd asdf #\" adf ++--// /*\" ";
if (rx.IsMatch(test))
{
Console.WriteLine(" -->| " + rx.ToString() + " | matched: " + test);
}
else
{
Console.WriteLine("No match");
}
}
} // class
} // namespace
Here's how it works:
the program has a set of the file extensions you want to analyze.
It walks through each extension in the set, getting all files of that type in the current and all subdirectories.
It selects each file, goes through each line of that file, compares each line to a regex to figure out what it's looking at, and increments the line count after it figures out what it's looking at.
If a line isn't whitespace, a single or multi-line comment, or a multi-line string, it counts it as a line of code. It reports all the counts for each of those types of lines (code, comments, whitespace, multi-line strings) and writes them to a CSV file. No need to explain why Visual Studio did or did not count something as a line of code.
Yes, there are three loops embedded in each other (O(n-cubed) O_O ) but it's just a simple, standalone developer tool, and the biggest code base I've run it on was about 350K lines and it took like 10 seconds to run on a Core i7.
Edit: Just ran it on the Firefox 12 code base, about 4.3 million lines (3.3M code, 1M comments), about 21K files, with an AMD Phenom processor - took 7 minutes, watched the performance tab in Task Manager, no stress. FYI.
My attitude is if I wrote it to be part of an instruction fed to a compiler, it's a line of code and should be counted.
It can easily be customized to ignore or count whatever you want (brackets, namespaces, the includes at the top of the file, etc). Just add the regex, test it with the function that's right there below the regexes, then update the if statement with that regex.

Creating a unique filename from a list of alphanumeric strings

I apologize for creating a similar thread to many that are out there now, but I mainly wanted to also get some insight on some methods.
I have a list of Strings (could be just 1 or over a 1000)
Format = XXX-XXXXX-XX where each one is alphanumeric
I am trying to generate a unique string (currently 18 in length but probably could be longer ensuring not to maximize file length or path length) that I could reproduce if I have that same list. Order doesn't matter; although I may be interested if its easier to restrict the order as well.
My current Java code is follows (which failed today, hence why I am here):
public String createOutputFileName(ArrayList alInput, EnumFPFunction efpf, boolean pHeaders) {
/* create file name based on input list */
String sFileName = "";
long partNum = 0;
for (String sGPN : alInput) {
sGPN = sGPN.replaceAll("-", ""); //remove dashes
partNum += Long.parseLong(sGPN, 36); //(base 36)
}
sFileName = Long.toString(partNum);
if (sFileName.length() > 19) {
sFileName.substring(0, 18); //Max length of 19
}
return alInput;
}
So obviously just adding them did not work out so well I found out (also think I should take last 18 digits and not first 18)
Are there any good methods out there (possibly CRC related) that would work?
To assist with my key creation:
The first 3 characters are almost always numeric and would probably have many duplicate (out of 100, there may only be 10 different starting numbers)
These characters are not allowed - I,O
There will never be a character then a number in the last two alphachar subset.
I would use the system time. Here's how you might do it in Java:
public String createOutputFileName() {
long mills = System.currentTimeMillis();
long nanos = System.nanoTime();
return mills + " " + nanos;
}
If you want to add some information about the items and their part numbers, you can, of course!
======== EDIT: "What do I mean by batch object" =========
class Batch {
ArrayList<Item> itemsToProcess;
String inputFilename; // input to external process
boolean processingFinished;
public Batch(ArrayList<Item> itemsToProcess) {
this.itemsToProcess = itemsToProcess;
inputFilename = null;
processingFinished = false;
}
public void processWithExternal() {
if(inputFilename != null || processingFinished) {
throw new IllegalStateException("Cannot initiate process more than once!");
}
String base = System.currentTimeMillis() + " " + System.nanoTime();
this.inputFilename = base + "_input";
writeItemsToFile();
// however you build your process, do it here
Process p = new ProcessBuilder("myProcess","myargs", inputFilename);
p.start();
p.waitFor();
processingFinished = true;
}
private void writeItemsToFile() {
PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(inputFilename)));
int flushcount = 0;
for(Item item : itemsToProcess) {
String output = item.getFileRepresentation();
out.println(output);
if(++flushcount % 10 == 0) out.flush();
}
out.flush();
out.close();
}
}
In addition to GlowCoder's response, I have thought of another "decent one" that would work.
Instead of just adding the list in base 36, I would do two separate things to the same list.
In this case, since there is no way for negative or decimal numbers, adding every number and multiplying every number separately and concatenating these base36 number strings isn't a bad way either.
In my case, I would take the last nine digits of the added number and last nine of the multiplied number. This would eliminate my previous errors and make it quite robust. It obviously is still possible for errors once overflow starts occurring, but could also work in this case. Extending the allowable string length would make it more robust as well.
Sample code:
public String createOutputFileName(ArrayList alInput, EnumFPFunction efpf, boolean pHeaders) {
/* create file name based on input list */
String sFileName1 = "";
String sFileName2 = "";
long partNum1 = 0; // Starting point for addition
long partNum2 = 1; // Starting point for multiplication
for (String sGPN : alInput) {
//remove dashes
sGPN = sGPN.replaceAll("-", "");
partNum1 += Long.parseLong(sGPN, 36); //(base 36)
partNum2 *= Long.parseLong(sGPN, 36); //(base 36)
}
// Initial strings
sFileName1 = "000000000" + Long.toString(partNum1, 36); // base 36
sFileName2 = "000000000" + Long.toString(partNum2, 36); // base 36
// Cropped strings
sFileName1 = sFileName1.substring(sFileName1.length()-9, sFileName1.length());
sFileName2 = sFileName2.substring(sFileName2.length()-9, sFileName2.length());
return sFileName1 + sFileName2;
}

Resources