I ran into some weird threading issues in a continuous integration build on travis-ci and got:
testThatDifferentArgumentsCanBeParsedConcurrently(se.softhouse.jargo.concurrency.ConcurrencyTest) Time elapsed: 0.479 sec <<< FAILURE!
org.junit.ComparisonFailure: expected:<... greeting phrase to [
]greet new connection...> but was:<... greeting phrase to []greet new connection...>:<... greeting phrase to []greet new connection...>
It only seems to go wrong with openjdk and not with oraclejdk.
And here's my code that's using BreakIterator:
/**
* Wraps lines where <a
* href="http://docs.oracle.com/javase/tutorial/i18n/text/line.html">appropriate</a> (as defined
* by {#code locale}).
*
* #param value the value to separate with {#link StringsUtil#NEWLINE new lines}
* #param startingIndex the index where each line starts, useful for a fixed-size table for
* instance
* #param maxLineLength how long each line are allowed to be
*/
public static StringBuilder wrap(CharSequence value, int startingIndex, int maxLineLength, Locale locale)
{
String textToSplit = value.toString();
StringBuilder result = new StringBuilder(textToSplit.length());
// TODO(jontejj): is this not thread safe?
BreakIterator boundary = BreakIterator.getLineInstance(locale);
boundary.setText(textToSplit);
int start = boundary.first();
int end = boundary.next();
int lineLength = startingIndex;
while(end != BreakIterator.DONE)
{
String word = textToSplit.substring(start, end);
lineLength = lineLength + word.length();
if(lineLength >= maxLineLength)
{
result.append(NEWLINE);
lineLength = startingIndex;
}
result.append(word);
start = end;
end = boundary.next();
}
return result;
}
It seems like java.text.BreakIterator didn't do it's job and stretched the line for more characters than it should have.
I checked the openjdk version of both BreakIterator and RuleBasedBreakIterator but couldn't identify any apparent thread safety issues. There are some arrays in RuleBasedBreakIterator that aren't deeply cloned but I couldn't see that they are modified so that should be safe. I could not find the oraclejdk source code. Have I understood it wrong that there shouldn't be as big differences between them as there used to be?
I tried to isolate the issue with a specific test for BreakIterator usage but so far that hasn't been fruitful.
Sorry for the kind of awkward question but I'm at a standstill so I thought that maybe someone on stackoverflow could help.
Related
EDIT:
I have heavily edited this question after making some significant new discoveries and the question not having any answers yet.
Historically/AFAIK, keeping your Mac awake while in closed-display mode and not meeting Apple's requirements, has only been possible with a kernel extension (kext), or a command run as root. Recently however, I have discovered that there must be another way. I could really use some help figuring out how to get this working for use in a (100% free, no IAP) sandboxed Mac App Store (MAS) compatible app.
I have confirmed that some other MAS apps are able to do this, and it looks like they might be writing YES to a key named clamshellSleepDisabled. Or perhaps there's some other trickery involved that causes the key value to be set to YES? I found the function in IOPMrootDomain.cpp:
void IOPMrootDomain::setDisableClamShellSleep( bool val )
{
if (gIOPMWorkLoop->inGate() == false) {
gIOPMWorkLoop->runAction(
OSMemberFunctionCast(IOWorkLoop::Action, this, &IOPMrootDomain::setDisableClamShellSleep),
(OSObject *)this,
(void *)val);
return;
}
else {
DLOG("setDisableClamShellSleep(%x)\n", (uint32_t) val);
if ( clamshellSleepDisabled != val )
{
clamshellSleepDisabled = val;
// If clamshellSleepDisabled is reset to 0, reevaluate if
// system need to go to sleep due to clamshell state
if ( !clamshellSleepDisabled && clamshellClosed)
handlePowerNotification(kLocalEvalClamshellCommand);
}
}
}
I'd like to give this a try and see if that's all it takes, but I don't really have any idea about how to go about calling this function. It's certainly not a part of the IOPMrootDomain documentation, and I can't seem to find any helpful example code for functions that are in the IOPMrootDomain documentation, such as setAggressiveness or setPMAssertionLevel. Here's some evidence of what's going on behind the scenes according to Console:
I've had a tiny bit of experience working with IOMProotDomain via adapting some of ControlPlane's source for another project, but I'm at a loss for how to get started on this. Any help would be greatly appreciated. Thank you!
EDIT:
With #pmdj's contribution/answer, this has been solved!
Full example project:
https://github.com/x74353/CDMManager
This ended up being surprisingly simple/straightforward:
1. Import header:
#import <IOKit/pwr_mgt/IOPMLib.h>
2. Add this function in your implementation file:
IOReturn RootDomain_SetDisableClamShellSleep (io_connect_t root_domain_connection, bool disable)
{
uint32_t num_outputs = 0;
uint32_t input_count = 1;
uint64_t input[input_count];
input[0] = (uint64_t) { disable ? 1 : 0 };
return IOConnectCallScalarMethod(root_domain_connection, kPMSetClamshellSleepState, input, input_count, NULL, &num_outputs);
}
3. Use the following to call the above function from somewhere else in your implementation:
io_connect_t connection = IO_OBJECT_NULL;
io_service_t pmRootDomain = IOServiceGetMatchingService(kIOMasterPortDefault, IOServiceMatching("IOPMrootDomain"));
IOServiceOpen (pmRootDomain, current_task(), 0, &connection);
// 'enable' is a bool you should assign a YES or NO value to prior to making this call
RootDomain_SetDisableClamShellSleep(connection, enable);
IOServiceClose(connection);
I have no personal experience with the PM root domain, but I do have extensive experience with IOKit, so here goes:
You want IOPMrootDomain::setDisableClamShellSleep() to be called.
A code search for sites calling setDisableClamShellSleep() quickly reveals a location in RootDomainUserClient::externalMethod(), in the file iokit/Kernel/RootDomainUserClient.cpp. This is certainly promising, as externalMethod() is what gets called in response to user space programs calling the IOConnectCall*() family of functions.
Let's dig in:
IOReturn RootDomainUserClient::externalMethod(
uint32_t selector,
IOExternalMethodArguments * arguments,
IOExternalMethodDispatch * dispatch __unused,
OSObject * target __unused,
void * reference __unused )
{
IOReturn ret = kIOReturnBadArgument;
switch (selector)
{
…
…
…
case kPMSetClamshellSleepState:
fOwner->setDisableClamShellSleep(arguments->scalarInput[0] ? true : false);
ret = kIOReturnSuccess;
break;
…
So, to invoke setDisableClamShellSleep() you'll need to:
Open a user client connection to IOPMrootDomain. This looks straightforward, because:
Upon inspection, IOPMrootDomain has an IOUserClientClass property of RootDomainUserClient, so IOServiceOpen() from user space will by default create an RootDomainUserClient instance.
IOPMrootDomain does not override the newUserClient member function, so there are no access controls there.
RootDomainUserClient::initWithTask() does not appear to place any restrictions (e.g. root user, code signing) on the connecting user space process.
So it should simply be a case of running this code in your program:
io_connect_t connection = IO_OBJECT_NULL;
IOReturn ret = IOServiceOpen(
root_domain_service,
current_task(),
0, // user client type, ignored
&connection);
Call the appropriate external method.
From the code excerpt earlier on, we know that the selector must be kPMSetClamshellSleepState.
arguments->scalarInput[0] being zero will call setDisableClamShellSleep(false), while a nonzero value will call setDisableClamShellSleep(true).
This amounts to:
IOReturn RootDomain_SetDisableClamShellSleep(io_connect_t root_domain_connection, bool disable)
{
uint32_t num_outputs = 0;
uint64_t inputs[] = { disable ? 1 : 0 };
return IOConnectCallScalarMethod(
root_domain_connection, kPMSetClamshellSleepState,
&inputs, 1, // 1 = length of array 'inputs'
NULL, &num_outputs);
}
When you're done with your io_connect_t handle, don't forget to IOServiceClose() it.
This should let you toggle clamshell sleep on or off. Note that there does not appear to be any provision for automatically resetting the value to its original state, so if your program crashes or exits without cleaning up after itself, whatever state was last set will remain. This might not be great from a user experience perspective, so perhaps try to defend against it somehow, for example in a crash handler.
This is general question, how to approach this problem.
For my technical degree i would like to do sort of website application that will connect windows machine, send a request to powershell e.g. get-processes, and in the end display it on the website.
I'm not sure if PowerShell Web Access can be modified like that, Is there any other solution?
Like service that i could communicate on?
-mateusz
You can use Powershell runspaces, this is an example, but in your case you might have to change it for the authentication methods in you have to use...
PSCredential credential = new PSCredential(user, secure_pw);
WSManConnectionInfo connectionInfo = new WSManConnectionInfo();
connectionInfo.AuthenticationMechanism = AuthenticationMechanism.Credssp;
connectionInfo.ProxyAuthentication = AuthenticationMechanism.Negotiate;
connectionInfo.OperationTimeout = 4 * 60 * 1000; // 4 minutes.
connectionInfo.OpenTimeout = 1 * 60 * 1000; // 1 minute.
connectionInfo.Credential = credential;
Runspace rs = RunspaceFactory.CreateRunspace(connectionInfo);
rs.Open();
using (PowerShell PowerShellInstance = PowerShell.Create())
{
string hostname = "my-host";
PowerShellInstance.Runspace = rs;
PowerShellInstance.AddScript(string.Format("param([string]$hostname) Get-Process -ComputerName $hostname"))
PowerShellInstance.AddParameter("hostname", hostname);
// invoke execution on the pipeline (collecting output)
Collection<PSObject> PSOutput = PowerShellInstance.Invoke();
// do something with the errors found.
if (PowerShellInstance.Streams.Error.Count > 0)
{
foreach (var error in PowerShellInstance.Streams.Error)
{
Console.WriteLine(error.Exception.Message);
}
}
}
rs.Dispose();
If do it this way, I recommend you do a bit of research about PowerShellInstance.AddScript vs PowerShellInstance.AddCommand and how the parameters have to be handled, etc...
The information in the MVStore docs on backing up a database is a little vague, and I'm not familiar with all the concepts and terminology, so I wanted to see if the approach I came up with makes sense.
I'm a Clojure programmer, so please forgive my Java here:
// db is an MVStore instance
FileStore fs = db.getFileStore();
FileOutputStream fos = java.io.FileOutputStream(pathToBackupFile);
FileChannel outChannel = fos.getChannel();
try {
db.commit();
db.setReuseSpace(false);
ByteBuffer bb = fs.readFully(0, fs.size());
outChannel.write(bb);
}
finally {
outChannel.close();
db.setReuseSpace(true);
}
Here's what it looks like in Clojure in case my Java is bad:
(defn backup-db
[db path-to-backup-file]
(let [fs (.getFileStore db)
backup-file (java.io.FileOutputStream. path-to-backup-file)
out-channel (.getChannel backup-file)]
(try
(.commit db)
(.setReuseSpace db false)
(let [file-contents (.readFully fs 0 (.size fs))]
(.write out-channel file-contents))
(finally
(.close out-channel)
(.setReuseSpace db true)))))
My approach seems to work, but I wanted to make sure I'm not missing anything or see if there's a better way. Thanks!
P.S. I used the H2 tag because MVStore doesn't exist and I don't have enough reputation to create it.
The docs currently say:
The persisted data can be backed up at any time, even during write
operations (online backup). To do that, automatic disk space reuse
needs to be first disabled, so that new data is always appended at the
end of the file. Then, the file can be copied. The file handle is
available to the application. It is recommended to use the utility
class FileChannelInputStream to do this.
The classes FileChannelInputStream and FileChannelOutputStream convert a java.nio.FileChannel into a standard InputStream and OutputStream. There is existing H2 code in BackupCommand.java that shows how to use them. We can improve upon it using Java 9 input.transferTo(output); to copy the data:
public void backup(MVStore s, File backupFile) throws Exception {
try {
s.commit();
s.setReuseSpace(false);
try(RandomAccessFile outFile = new java.io.RandomAccessFile(backupFile, "rw");
FileChannelOutputStream output = new FileChannelOutputStream(outFile.getChannel(), false)){
try(FileChannelInputStream input = new FileChannelInputStream(s.getFileStore().getFile(), false)){
input.transferTo(output);
}
}
} finally {
s.setReuseSpace(true);
}
}
Note that when you create the FileChannelInputStream you have to pass false to tell it to not close the underlying file channel when the stream is closed. If you don't do that it will close the file that your FileStore is trying to use. That code uses try-with-resource syntax to make sure that the output file is properly closed.
In order to try this, I checked out the mvstore code then modified the TestMVStore to add a testBackup() method which is similar to the existing testSimple() code:
private void testBackup() throws Exception {
// write some records like testSimple
String fileName = getBaseDir() + "/" + getTestName();
FileUtils.delete(fileName);
MVStore s = openStore(fileName);
MVMap<Integer, String> m = s.openMap("data");
for (int i = 0; i < 3; i++) {
m.put(i, "hello " + i);
}
// create a backup
String fileNameBackup = getBaseDir() + "/" + getTestName() + ".backup";
FileUtils.delete(fileNameBackup);
backup(s, new File(fileNameBackup));
// this throws if you accidentally close the input channel you get from the store
s.close();
// open the backup and verify
s = openStore(fileNameBackup);
m = s.openMap("data");
for (int i = 0; i < 3; i++) {
assertEquals("hello " + i, m.get(i));
}
s.close();
}
With your example, you are reading into a ByteBuffer which must fit into memory. Using the stream transferTo method uses an internal buffer that is currently (as at Java11) set to 8192 bytes.
I'm using MaxEntTagger for pos-tagging and sentence splitting by using the follwing codes:
MaxentTagger tagger = new MaxentTagger("models/left3words-wsj-0-18.tagger");
#SuppressWarnings("unchecked")
List<Sentence<? extends HasWord>> sentences = MaxentTagger.tokenizeText(new BufferedReader(new StringReader(out2)));
for (Sentence<? extends HasWord> sentence : sentences) {
content.append(sentence + "\n");
Sentence<TaggedWord> tSentence = MaxentTagger.tagSentence(sentence);
out.append(tSentence.toString(false) + "\n");
}
The problem is it will complain there are untokenizable characters in the text. And the tagged output will omit those untokenizable characters. So for example, the original text is:
Let Σ be a finite set of function symbols, the signature.
where Σ is in big5 code. But the program will show the following warning message:
Untokenizable: Σ (first char in decimal: 931)
and the tagged output is:
Let/VB be/VB a/DT finite/JJ set/NN of/IN function/NN symbols/NNS ,/, the/DT signature/NN ./.
the splitted sentence I got is:
Let be a finite set of function symbols , the signature .
My question is how to retain these untokenizable characters?
I've tried modifying the mode's props file but with no luck:
tagger training invoked at Sun Sep 21 23:03:26 PDT 2008 with arguments:
model = left3words-wsj-0-18.tagger
arch = left3words,naacl2003unknowns,wordshapes(3)
trainFile = /u/nlp/data/pos-tagger/train-wsj-0-18 ...
encoding = Big5
initFromTrees = false
Any suggestion?
Thanks Prof. Manning's help. But I encounter the same issue when utilizing parser tree.
The sequel
I need to get the parser tree of a sentence, so I used the following codes:
PTBTokenizer<Word> ptb = PTBTokenizer.newPTBTokenizer(new StringReader(sentences));
List<Word> words = ptb.tokenize();
Tree parseTree2 = lp.apply(words);
TreebankLanguagePack tlp = new PennTreebankLanguagePack();
GrammaticalStructureFactory gsf = tlp.grammaticalStructureFactory();
GrammaticalStructure gs = gsf.newGrammaticalStructure(parseTree2);
But I don't know how to set PTBTokenizer for resolving the issue of untokenizable characters this time.
If using the factory method to generate an PTBTokenizer object, I don't know how to concatenate it to the StringReader.
List<Word> words = ptb.getTokenizer(new StringReader(sentences));
doesn't work.
The Stanford tokenizer accepts a variety of options to control tokenization, including how characters it doesn't know about are handled. However, to set them, you currently have to instantiate your own tokenizer. But that's not much more difficult than what you have above. The following complete program makes a tokenizer with options and then tags using it.
The "noneKeep" option means that it logs no messages about unknown characters but keeps them and turns each into a single character token. You can learn about the other options in the PTBTokenizer class javadoc.
NOTE: you seem to be using a rather old version of the tagger. (We got rid of the Sentence class and started just using List's of tokens about 2 years ago, probably around the same time these options were added to the tokenizer.) So you may well have to upgrade to the latest version. At any rate, the code below will only compile correctly against a more recent version of the tagger.
import java.io.*;
import java.util.*;
import edu.stanford.nlp.ling.*;
import edu.stanford.nlp.process.*;
import edu.stanford.nlp.objectbank.TokenizerFactory;
import edu.stanford.nlp.tagger.maxent.MaxentTagger;
/** This demo shows user-provided sentences (i.e., {#code List<HasWord>})
* being tagged by the tagger. The sentences are generated by direct use
* of the DocumentPreprocessor class.
*/
class TaggerDemo2 {
public static void main(String[] args) throws Exception {
if (args.length != 2) {
System.err.println("usage: java TaggerDemo modelFile fileToTag");
return;
}
MaxentTagger tagger = new MaxentTagger(args[0]);
TokenizerFactory<CoreLabel> ptbTokenizerFactory =
PTBTokenizer.factory(new CoreLabelTokenFactory(), "untokenizable=noneKeep");
BufferedReader r =
new BufferedReader(new InputStreamReader(new FileInputStream(args[1]), "utf-8"));
PrintWriter pw = new PrintWriter(new OutputStreamWriter(System.out, "utf-8"));
DocumentPreprocessor documentPreprocessor = new DocumentPreprocessor(r);
documentPreprocessor.setTokenizerFactory(ptbTokenizerFactory);
for (List<HasWord> sentence : documentPreprocessor) {
List<TaggedWord> tSentence = tagger.tagSentence(sentence);
pw.println(Sentence.listToString(tSentence, false));
}
}
}
I'm using Joomla 2.5 to build a medium-sized website, and I've decided to ease maintenance and content management headaches through creating menus automatically.
I've looked for extensions which did this, but only found Joomla 1.5 extensions. I ended up trying to upgrade a GPL extension called Auto Menu Magic.
It was easy to deal with the basic issues like the XML tags in the extension file, since there's a page in the Joomla which helps you migrate from Joomla 1.5 to 1.6.
The extension I mentioned has a function named onContentAfterSave which is called by joomla when an article is saved. I've been debugging through creating rubbish articles in the admin interface and changing the code to throw exceptions in several places, which I can see as error messages on the admin frontend.
This is where I got stuck:
$db = &JFactory::getDBO();
$menu = JTable::getInstance( 'menu');
$menu->menutype = $menutype;
$menu->name = $name;
$menu->link = $link;
$menu->type = $linktype;
$menu->published = $published;
$menu->componentid = $componentid;
$menu->parent = $menuparentid;
$menu->sublevel = $menusublevel;
$menu->checked_out = 0;
$menu->checked_out_time = 0;
$menu->pollid = 0;
$menu->browserNav = 0;
$menu->access = 0;
$menu->utaccess = 0;
$menu->lft = 0;
$menu->rgt = 0;
$menu->home = 0;
$menu->params = $params;
// Figure out the order (Just pop this article at the end of the list):
$menu->ordering = $menu->getNextOrder(
"menutype = ".$db->Quote($menu->menutype).
" AND published >= 0 AND parent = ".(int) $menu->parent
);
// Validate:
if (!$menu->check())
return NULL;
// DEBUG 2 -- Integrity check
throw new Exception ("menutype: $menutype, name: $name, link: $link published: $published, componentid: $componentid menuparentid: $menuparentid menusublevel: $menusublevel");
// Save:
if (!$menu->store())
return NULL;
// DEBUG 1
throw new Exception(" Could save! ");
As you can see above, I tried to throw an exception (DEBUG 1) when the menu was saved to the database. This exception was never reached, but the upper exception (DEBUG 2) is reached. This means that $menu->check() returns true, but not $menu->store(). I assume that the database is returning an error because some of the Joomla database structure might have changed after 1.5.
I have read the source a lot these past hours, but I can't find one thing. How can I look at the columns that a Joomla table uses, so I can debug this error properly?
Thanks in advance!
PS: I've looked at the SQL database too, but it doesn't help much. The variables seem to have different naming conventions from the column names.
I think it should look like this because I have been trying to convert auto menu as well!
$db = &JFactory::getDBO();
$menu = JTable::getInstance('menu');
$menu->menutype = $menutype;
$menu->title = $title;
$menu->alias = strtolower($title) ;
$menu->note = "Created by automenu";
$menu->path = $link;
$menu->link = $link;
$menu->type = $linktype;
$menu->published = $published;
$menu->parent_id = $menuparentid
$menu->level = $menusublevel;
$menu->componentid = $componentid;
$menu->ordering = 0;
$menu->checked_out = 0;
$menu->checked_out_time = 0;
$menu->browserNav = 0;
$menu->access = 1;
$menu->img = '';
$menu->templat_style_id = 0;
$menu->params = $params;
$menu->lft = 0;
$menu->rgt = 0;
$menu->home = 0;
$menu->language = '*';
$menu->client_id = 0;
I would be interseted to know if you ever got it working!
I'd suggest turning on Joomla debugging in the System Configuration. At the bottom of each page it shows all the queries it has executed, and this (depending on the plugin) might show you what SQL is being executed, and presumably, failing. There's likely to be a big list, so you may have to search through it a bit to find the statement you're interested in.
Fabio,
Many thanks! I will try it out and see if I can improve it further.
Mike
You're forgetting the first rule of Exceptions Club, if you throw something... you have to catch it.
I don't see a try/catch pair in your code so PHP would be stopping with a "Fatal Error..." for the uncaught exception so it would never get to the DEBUG 1. e.g.
Fatal error: Uncaught exception 'Exception' with message ...
Try wrapping your code in a try/catch pair and allowing the execution to continue after DEBUG 2 have a look at the PHP doc's for exceptions