chronicle-bytes shared DirectBytesStores - chronicle

I've created a MappedBytes instance to a file that I'm using as shared cache between different Java processes.
I would like to be able to split out additional MappedByte instances (or ByteBuffer or any other instance) from the original that provide direct read/write access to a subset of the underlying file.
I've spent today experimenting with different methods but options like subBytes(), rawCopy() and copyTo() all seem to create local copies of the underlying file, rather than accessing the file directly.
For example:
File tmpFile = new File(System.getProperty("java.io.tmpdir"), "data.dat");
MappedFile mappedFile = MappedFile.mappedFile(tmpfile, 1000, 100, 10, false);
MappedBytes original = MappedBytes.mappedBytes(mappedFile);
original.zeroOut(0, 1000);
original.writeInt(0, 1234);
BytesStore copy = original.bytesStore().subBytes(0, 200);
// Print out the int in the two BytesStores.
// This shows that the copy has the same contents of the original.
System.out.println("Original(0): " + original.readInt(0));
System.out.println("Copy(0): " + copy.readInt(0));
// Now modify the copy and print out the new int in the two BytesStores again.
copy.writeInt(50, 4321);
System.out.println("Original(50): " + original.readInt(50));
System.out.println("Copy(50): " + copy.readInt(50));
Produces the output:
Original(0): 1234
Copy(0): 1234
Original(50): 0
Copy(50): 4321
The copy has been modified but not the original. I would like the original to be modified, can chronicle-bytes do this?
Thanks for your help,
Josh.

This is a self-contained test which I think behaves the way you need.
#Test
public void multiBytes() throws FileNotFoundException {
String tmpfile = OS.TMP + "/data.dat";
MappedFile mappedFile = MappedFile.mappedFile(new File(tmpfile), 64 << 10);
MappedBytes original = MappedBytes.mappedBytes(mappedFile);
original.zeroOut(0, 1000);
original.writeInt(0, 1234);
PointerBytesStore pbs = new PointerBytesStore();
pbs.set(original.addressForRead(50), 100);
// Print out the int in the two BytesStores.
// This shows that the copy has the same contents of the original.
System.out.println("Original(0): " + original.readInt(0));
System.out.println("PBS(0): " + pbs.readInt(0));
// Now modify the copy and print out the new int in the two BytesStores again.
pbs.writeInt(0, 4321);
System.out.println("Original(50): " + original.readInt(50));
System.out.println("PBS(0): " + pbs.readInt(0));
original.writeInt(54, 12345678);
System.out.println("Original(54): " + original.readInt(54));
System.out.println("PBS(4): " + pbs.readInt(4));
}
prints
Original(0): 1234
PBS(0): 0
Original(50): 4321
PBS(0): 4321
Original(54): 12345678
PBS(4): 12345678

Related

Split window java script for InDesign

Does anyone know of or can help me figure out how to create a script for InDesign to do the Split Screen command, this is found under menu selection Window > Arrange > Split Window. Thank you!
Basically this should work:
app.menuActions.item("$ID/Split Window").invoke()
But actually it sometimes works and sometimes doesn't. I can't tell why.
It works more or less stable as a toggle split/unsplit this way:
try {
app.menuActions.item("$ID/Split Window").invoke()
} catch(e) {
app.menuActions.item("$ID/Unsplit Window").invoke()
}
Probably there is some property, which you can read and run split command only if the window is not splitted already. I don't know.
Update
The list of 'menuActions' you can get with this script:
var list = "ID\tName\tArea\n";
var i = 0;
for (var i=0; i<app.menuActions.length; i++) {
var m = app.menuActions[i];
list += m.id + "\t" + m.name + "\t" + m.area + "\n";
}
f = File("menu_actions.txt");
f.encoding = "UTF-8";
f.open("w");
f.write(list);
f.close();
f.execute(); // to open the txt file immediately
\\ alert(i + " actions wese saved to " + f.fsName);
All credits to Kasyan Servetsky: http://kasyan.ho.ua/tips/indesign_script/all/open_menu_item.html

Image Transmit to Intermec PM4i printer and then Print

I'm using Fingerprint to upload and then print image with pcx format.
Step1 Upload image to printer using TCP port, I use command :
IMAGE LOAD "bigfoot.1",1746,""\r\n
The printer returns with message "OK".
And then I send bytes data of bigfoot.1 to printer using socket.
Step 2 Print the image "bigfoot.1":
PRPOS 200,200
DIR 3
ALIGN 5
PRIMAGE "bigfoot.1"
PRINTFEED
RUN
The problem comes, the printer returns with message "Image not found". So I come up with the possibility of failure of upload. So I open the software PrintSet4 to check the image, the image already exists in TMP.Odd!!!
At last, I used PrintSet4 to substitute my socket application to upload image, After add file and apply, I use the step2 print command to print image, It works fine!
Here is the C# code to upload Image:
public void SendFile(string filePath, string CR_LF)
{
FileInfo fi = new FileInfo(filePath);
using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
byte[] byteFile = new byte[fs.Length];
string cmd = "IMAGE LOAD \"" + fi.Name + "\"," + byteFile.Length.ToString() + ",\" \"" + CR_LF;
ClientSocket.Send(encode.GetBytes(cmd));
fs.Read(byteFile, 0, byteFile.Length);
Thread.Sleep(1000);
ClientSocket.Send(byteFile);
}
}
I have modified your code and used serial port.
public void SendFile(string filePath)
{
SerialPort port = new SerialPort("COM3", 38400, Parity.None, 8, StopBits.One);
port.Open();
FileInfo fi = new FileInfo(filePath);
using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
byte[] byteFile = new byte[fs.Length];
// string cmd = "IMAGE LOAD \"" + fi.Name + "\"," + teFile.Length.ToString()+ ",\"\"" + CR_LF;
string cmd = "IMAGE LOAD " + "\"" + fi.Name + "\"" + "," + byteFile.Length.ToString() + "," + "\"S\"";
port.WriteLine(cmd);
fs.Read(byteFile, 0, byteFile.Length);
port.Write(byteFile,0,byteFile.Count());
int count = byteFile.Count();
int length = byteFile.Length;
}
}
So I noticed the problem was using CR_LF. Instead, I used port.WriteLine(cmd), which acts the same as adding a line separator. And it worked fine.

Script Works on Win 7, Not on Server 2003

I have a script that is rather simple, it boots up WinSCP and checks the directory for a file that starts with "TSA". If the file exists, it exits, if it does not exist, it transfers over a new file.
Its up and running on my Windows 7 machine, that is where i created it - but when i transfer it over to my server [windows server 2003] it never finds the file.
My script:
var FILEPATH = "../zfinance/TSA";
// Session to connect to
var SESSION = "someplace#somewhere.com";
// Path to winscp.com
var WINSCP = "c:\\program files\\winscp\\winscp.com";
var filesys = WScript.CreateObject("Scripting.FileSystemObject");
var shell = WScript.CreateObject("WScript.Shell");
var logfilepath = filesys.GetSpecialFolder(2) + "\\" + filesys.GetTempName() + ".xml";
var p = FILEPATH.lastIndexOf('/');
var path = FILEPATH.substring(0, p);
var filename = FILEPATH.substring(p + 1);
var exec;
// run winscp to check for file existence
exec = shell.Exec("\"" + WINSCP + "\" /log=\"" + logfilepath + "\"");
exec.StdIn.Write(
"option batch abort\n" +
"open \"" + SESSION + "\"\n" +
"ls \"" + path + "\"\n" +
"exit\n");
// wait until the script finishes
while (exec.Status == 0)
{
WScript.Sleep(100);
WScript.Echo(exec.StdOut.ReadAll());
}
if (exec.ExitCode != 0)
{
WScript.Echo("Error checking for file existence");
WScript.Quit(1);
}
// look for log file
var logfile = filesys.GetFile(logfilepath);
if (logfile == null)
{
WScript.Echo("Cannot find log file");
WScript.Quit(1);
}
// parse XML log file
var doc = new ActiveXObject("MSXML2.DOMDocument");
doc.async = false;
doc.load(logfilepath);
doc.setProperty("SelectionNamespaces",
"xmlns:w='http://winscp.net/schema/session/1.0'");
doc.setProperty("SelectionLanguage", "XPath");
var nodes = doc.selectNodes("//w:file/w:filename[starts-with(#value, '" + filename + "')]");
if (nodes.length > 0)
{
WScript.Echo("File found");
WScript.Quit(0);
}
else
{
WScript.Echo("File not found");
WScript.Quit(1);
}
After much investigation, i think i've found the piece of code that does not function properly:
// parse XML log file
var doc = new ActiveXObject("MSXML2.DOMDocument.6.0");
doc.async = false;
doc.load(logfilepath);
doc.setProperty("SelectionNamespaces",
"xmlns:w='http://winscp.net/schema/session/1.0'");
The only problem is, i have no idea why. The log file at this point should be written over with the xml code, but this does not happen.
Thanks in advance for any help.
And the answer is........... WinSCP on Windows Server 2003 was WAY out of date. So out of date that the log was completely different from one version to the next. Updated and VIOLA! Problem solved. Thanks for your help.
Maybe you need to install MSXML2.DOMDocument.6.0
http://msdn.microsoft.com/en-us/library/windows/desktop/cc507436%28v=vs.85%29.aspx
If you open up regedit and look for "MSXML2.DOMDocument.6.0" does it find it? If so maybe security settings for the script need to be set in order to be able to create an activeX object.
What can you see when you put some stuff in try catch?
try{
//stuff
}catch(e){
WScript.Echo(e.message);
}

Visual Studio code metrics misreporting lines of code

The code metrics analyser in Visual Studio, as well as the code metrics power tool, report the number of lines of code in the TestMethod method of the following code as 8.
At the most, I would expect it to report lines of code as 3.
[TestClass]
public class UnitTest1
{
private void Test(out string str)
{
str = null;
}
[TestMethod]
public void TestMethod()
{
var mock = new Mock<UnitTest1>();
string str;
mock.Verify(m => m.Test(out str));
}
}
Can anyone explain why this is the case?
Further info
After a little more digging I've found that removing the out parameter from the Test method and updating the test code causes LOC to be reported as 2, which I believe is correct. The addition of out causes the jump, so it's not because of braces or attributes.
Decompiling the DLL with dotPeek reveals a fair amount of additional code generated because of the out parameter which could be considered 8 LOC, but removing the parameter and decompiling also reveals generated code, which could be considered 5 LOC, so it's not simply a matter of VS counting compiler generated code (which I don't believe it should do anyway).
There are several common definitions of 'Lines Of Code' (LOC). Each tries to bring some sense to what I think of as an almost meaningless metric. For example google of effective lines of code (eLOC).
I think that VS is including the attribute as part of the method declaration and is trying to give eLOC by counting statements and even braces. One possiblity is that 'm => m.Test(out str)' is being counted as a statement.
Consider this:
if (a > 1 &&
b > 2)
{
var result;
result = GetAValue();
return result;
}
and this:
if (a> 1 && b >2)
return GetAValue();
One definition of LOC is to count the lines that have any code. This may even include braces. In such an extreme simplistic definition the count varies hugely on coding style.
eLOC tries to reduce or eliminate the influence of code style. For example, as may the case here, a declaration may be counted as a 'line'. Not justifying it, just explaining.
Consider this:
int varA = 0;
varA = GetAValue();
and this:
var varA = GetAValue();
Two lines or one?
It all comes down to what is the intent. If it is to measure how tall a monitor you need then perhaps use a simple LOC. If the intent is to measure complexity then perhaps counting code statements is better such as eLOC.
If you want to measure complexity then use a complexity metric like cyclomatic complexity. Don't worry about how VS is measuring LOC as, i think, it is a useless metric anyway.
With the tool NDepend we get a # Lines of Code (LoC) of 2 for TestMethod(). (Disclaimer I am one of the developers of this tool). I wrote an article about How do you count your number of Lines Of Code (LOC) ? that is shedding light on what is logical LoC, and how all .NET LoC counting tooling rely on the PDB sequence points technology.
My guess concerning this LoC value of 8 provided by VS metric, is that it includes the LoC of the method generated by the lambda expression + it includes the PDB sequences points related to open/ending braces (which NDepend doesn't). Also lot of gymnastic is done by the compiler to do what is called capturing the local variable str, but this shouldn't impact the #LoC that is inferred from the PDB sequence points.
Btw, I wrote 2 others related LoC articles:
Why is it useful to count the number of Lines Of Code (LOC) ?
Mythical man month : 10 lines per developer day
I was wondering about the Visual Studio line counting and why what I was seeing wasn't what was being reported. So I wrote a small C# console program to count pure lines of code and write the results to a CSV file (see below).
Open a new solution, copy and paste it into the Program.cs file, build the executable, and then you're ready to go. It's a .Net 3.5 application. Copy it into the topmost directory of your code base. Open a command window and run the executable. You get two prompts, first for name of the program/subsystem, and for any extra file types you want to analyze. It then writes the results to a CSV file in the current directory. Nice simple thing for your purposes or to hand to management.
Anyhoo, here it is, FWIW, and YMMV:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.IO;
namespace CodeMetricsConsole
{
class Program
{
// Concept here is that the program has a list of file extensions to do line counts on; it
// gets any extra extensions at startup from the user. Then it gets a list of files based on
// each extension in the current directory and all subdirectories. Then it walks through
// each file line by line and will display counts for that file and for that file extension.
// It writes that information to a CSV file in the current directory. It uses regular expressions
// on each line of each file to figure out what it's looking at, and how to count it (i.e. is it
// a line of code, a single or multi line comment, a multi-line string, or a whitespace line).
//
static void Main(string[] args)
{
try
{
Console.WriteLine(); // spacing
// prompt user for subsystem or application name
String userInput_subSystemName;
Console.Write("Enter the name of this application or subsystem (required): ");
userInput_subSystemName = Console.ReadLine();
if (userInput_subSystemName.Length == 0)
{
Console.WriteLine("Application or subsystem name required, exiting.");
return;
}
Console.WriteLine(); // spacing
// prompt user for additional types
String userInput_additionalFileTypes;
Console.WriteLine("Default extensions are asax, css, cs, js, aspx, ascx, master, txt, jsp, java, php, bas");
Console.WriteLine("Enter a comma-separated list of additional file extensions (if any) you wish to analyze");
Console.Write(" --> ");
userInput_additionalFileTypes = Console.ReadLine();
// tell user processing is starting
Console.WriteLine();
Console.WriteLine("Getting LOC counts...");
Console.WriteLine();
// the default file types to analyze - hashset to avoid duplicates if the user supplies extensions
HashSet allowedExtensions = new HashSet { "asax", "css", "cs", "js", "aspx", "ascx", "master", "txt", "jsp", "java", "php", "bas" };
// Add user-supplied types to allowedExtensions if any
String[] additionalFileTypes;
String[] separator = { "," };
if (userInput_additionalFileTypes.Length > 0)
{
// split string into array of additional file types
additionalFileTypes = userInput_additionalFileTypes.Split(separator, StringSplitOptions.RemoveEmptyEntries);
// walk through user-provided file types and append to default file types
foreach (String ext in additionalFileTypes)
{
try
{
allowedExtensions.Add(ext.Trim()); // remove spaces
}
catch (Exception e)
{
Console.WriteLine("Exception: " + e.Message);
}
}
}
// summary file to write to
String summaryFile = userInput_subSystemName + "_Summary.csv";
String path = Directory.GetCurrentDirectory();
String pathAndFile = path + Path.DirectorySeparatorChar + summaryFile;
// regexes for the different line possibilities
Regex oneLineComment = new Regex(#"^\s*//"); // match whitespace to two slashes
Regex startBlockComment = new Regex(#"^\s*/\*.*"); // match whitespace to /*
Regex whiteSpaceOnly = new Regex(#"^\s*$"); // match whitespace only
Regex code = new Regex(#"\S*"); // match anything but whitespace
Regex endBlockComment = new Regex(#".*\*/"); // match anything and */ - only used after block comment detected
Regex oneLineBlockComment = new Regex(#"^\s*/\*.*\*/.*"); // match whitespace to /* ... */
Regex multiLineStringStart = new Regex("^[^\"]*#\".*"); // match #" - don't match "#"
Regex multiLineStringEnd = new Regex("^.*\".*"); // match double quotes - only used after multi line string start detected
Regex oneLineMLString = new Regex("^.*#\".*\""); // match #"..."
Regex vbaComment = new Regex(#"^\s*'"); // match whitespace to single quote
// Uncomment these two lines to test your regex with the function testRegex() below
//new Program().testRegex(oneLineMLString);
//return;
FileStream fs = null;
String line = null;
int codeLineCount = 0;
int commentLineCount = 0;
int wsLineCount = 0;
int multiLineStringCount = 0;
int fileCodeLineCount = 0;
int fileCommentLineCount = 0;
int fileWsLineCount = 0;
int fileMultiLineStringCount = 0;
Boolean inBlockComment = false;
Boolean inMultiLineString = false;
try
{
// write to summary CSV file, overwrite if exists, don't append
using (StreamWriter outFile = new StreamWriter(pathAndFile, false))
{
// outFile header
outFile.WriteLine("filename, codeLineCount, commentLineCount, wsLineCount, mlsLineCount");
// walk through files with specified extensions
foreach (String allowed_extension in allowedExtensions)
{
String extension = "*." + allowed_extension;
// reset accumulating values for extension
codeLineCount = 0;
commentLineCount = 0;
wsLineCount = 0;
multiLineStringCount = 0;
// Get all files in current directory and subdirectories with specified extension
String[] fileList = Directory.GetFiles(Directory.GetCurrentDirectory(), extension, SearchOption.AllDirectories);
// walk through all files of this type
for (int i = 0; i < fileList.Length; i++)
{
// reset values for this file
fileCodeLineCount = 0;
fileCommentLineCount = 0;
fileWsLineCount = 0;
fileMultiLineStringCount = 0;
inBlockComment = false;
inMultiLineString = false;
try
{
// open file
fs = new FileStream(fileList[i], FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
using (TextReader tr = new StreamReader(fs))
{
// walk through lines in file
while ((line = tr.ReadLine()) != null)
{
if (inBlockComment)
{
if (whiteSpaceOnly.IsMatch(line))
{
fileWsLineCount++;
}
else
{
fileCommentLineCount++;
}
if (endBlockComment.IsMatch(line)) inBlockComment = false;
}
else if (inMultiLineString)
{
fileMultiLineStringCount++;
if (multiLineStringEnd.IsMatch(line)) inMultiLineString = false;
}
else
{
// not in a block comment or multi-line string
if (oneLineComment.IsMatch(line))
{
fileCommentLineCount++;
}
else if (oneLineBlockComment.IsMatch(line))
{
fileCommentLineCount++;
}
else if ((startBlockComment.IsMatch(line)) && (!(oneLineBlockComment.IsMatch(line))))
{
fileCommentLineCount++;
inBlockComment = true;
}
else if (whiteSpaceOnly.IsMatch(line))
{
fileWsLineCount++;
}
else if (oneLineMLString.IsMatch(line))
{
fileCodeLineCount++;
}
else if ((multiLineStringStart.IsMatch(line)) && (!(oneLineMLString.IsMatch(line))))
{
fileCodeLineCount++;
inMultiLineString = true;
}
else if ((vbaComment.IsMatch(line)) && (allowed_extension.Equals("txt") || allowed_extension.Equals("bas"))
{
fileCommentLineCount++;
}
else
{
// none of the above, thus it is a code line
fileCodeLineCount++;
}
}
} // while
outFile.WriteLine(fileList[i] + ", " + fileCodeLineCount + ", " + fileCommentLineCount + ", " + fileWsLineCount + ", " + fileMultiLineStringCount);
fs.Close();
fs = null;
} // using
}
finally
{
if (fs != null) fs.Dispose();
}
// update accumulating values
codeLineCount = codeLineCount + fileCodeLineCount;
commentLineCount = commentLineCount + fileCommentLineCount;
wsLineCount = wsLineCount + fileWsLineCount;
multiLineStringCount = multiLineStringCount + fileMultiLineStringCount;
} // for (specific file)
outFile.WriteLine("Summary for: " + extension + ", " + codeLineCount + ", " + commentLineCount + ", " + wsLineCount + ", " + multiLineStringCount);
} // foreach (all files with specified extension)
} // using summary file streamwriter
Console.WriteLine("Analysis complete, file is: " + pathAndFile);
} // try block
catch (Exception e)
{
Console.WriteLine("Error: " + e.Message);
}
}
catch (Exception e2)
{
Console.WriteLine("Error: " + e2.Message);
}
} // main
// local testing function for debugging purposes
private void testRegex(Regex rx)
{
String test = " asdfasd asdf #\" adf ++--// /*\" ";
if (rx.IsMatch(test))
{
Console.WriteLine(" -->| " + rx.ToString() + " | matched: " + test);
}
else
{
Console.WriteLine("No match");
}
}
} // class
} // namespace
Here's how it works:
the program has a set of the file extensions you want to analyze.
It walks through each extension in the set, getting all files of that type in the current and all subdirectories.
It selects each file, goes through each line of that file, compares each line to a regex to figure out what it's looking at, and increments the line count after it figures out what it's looking at.
If a line isn't whitespace, a single or multi-line comment, or a multi-line string, it counts it as a line of code. It reports all the counts for each of those types of lines (code, comments, whitespace, multi-line strings) and writes them to a CSV file. No need to explain why Visual Studio did or did not count something as a line of code.
Yes, there are three loops embedded in each other (O(n-cubed) O_O ) but it's just a simple, standalone developer tool, and the biggest code base I've run it on was about 350K lines and it took like 10 seconds to run on a Core i7.
Edit: Just ran it on the Firefox 12 code base, about 4.3 million lines (3.3M code, 1M comments), about 21K files, with an AMD Phenom processor - took 7 minutes, watched the performance tab in Task Manager, no stress. FYI.
My attitude is if I wrote it to be part of an instruction fed to a compiler, it's a line of code and should be counted.
It can easily be customized to ignore or count whatever you want (brackets, namespaces, the includes at the top of the file, etc). Just add the regex, test it with the function that's right there below the regexes, then update the if statement with that regex.

Why is the Scala-written lines deduplication app of mine so slow?

I've got some big (let's say 200 MiB - 2 GiB) textual files filled with tons of duplicated records. Each line can have about 100 or even more exact duplicates spread over the file. The task is to remove all the repetitions, leaving one unique instance of every record.
I've implemented it as follows:
object CleanFile {
def apply(s: String, t: String) {
import java.io.{PrintWriter, FileWriter, BufferedReader, FileReader}
println("Reading " + s + "...")
var linesRead = 0
val lines = new scala.collection.mutable.ArrayBuffer[String]()
val fr = new FileReader(s)
val br = new BufferedReader(fr)
var rl = ""
while (rl != null) {
rl = br.readLine()
if (!lines.contains(rl))
lines += rl
linesRead += 1
if (linesRead > 0 && linesRead % 100000 == 0)
println(linesRead + " lines read, " + lines.length + " unique found.")
}
br.close()
fr.close()
println(linesRead + " lines read, " + lines.length + " unique found.")
println("Writing " + t + "...")
val fw = new FileWriter(t);
val pw = new PrintWriter(fw);
lines.foreach(line => pw.println(line))
pw.close()
fw.close()
}
}
And it takes ~15 minutes (on my Core 2 Duo with 4 GB RAM) to process a 92 MiB file. While the following command:
awk '!seen[$0]++' filename
Takes about a minute to process 1.1 GiB file (which would take many hours with the above code of mine).
What's wrong with my code?
What is wrong is that you're using an array to store the lines. Lookup (lines.contains) takes O(n) in an array, so the whole thing runs in O(n²) time. By contrast, the Awk solution uses a hashtable, meaning O(1) lookup and a total running time of O(n).
Try using a mutable.HashSet instead.
You could also just read all lines and call .distinct on them. I don't know how distinct is implemented, but I'm betting it uses a HashSet to do it.

Resources