I am using the standard Ruby 2.2 Logger (i.e. not the Logger from Rails), and since I want to use it as rotating logger, I call it with shift_age and shift_time and pass to it a filename, not a open file handle, like this:
Logger.new('my_file.log', 4, 100_000)
I would like to make the output of this logger unbuffered, but I did not find any method in the Logger interface which would allow this.
In the particular way I am using this logger (I am wrapping the Logger with my own class, which does customization), I have one central point in my code where each logging request is going through, so if I could get at the underlying IO object, I could do a sync at this point, but I also don't know how to find this IO object.
Any ideas?
You do not need to set sync. It's already set by Logger upon logfile creation:
def create_logfile(filename)
begin
logdev = open(filename, (File::WRONLY | File::APPEND | File::CREAT | File::EXCL))
logdev.flock(File::LOCK_EX)
logdev.sync = true
add_log_header(logdev)
logdev.flock(File::LOCK_UN)
rescue Errno::EEXIST
# file is created by another process
logdev = open_logfile(filename)
logdev.sync = true
end
logdev
end
Source:
https://github.com/ruby/ruby/blob/trunk/lib/logger.rb
Related
I am currently working on a GUI using qt designer. I am wondering how I should go about printing strings on the GUI that acts like a logger window. I am using pyqt5.
Adapted from Todd Vanyo's example for PyQt5:
import sys
from PyQt5 import QtWidgets
import logging
# Uncomment below for terminal log messages
# logging.basicConfig(level=logging.DEBUG, format=' %(asctime)s - %(name)s - %(levelname)s - %(message)s')
class QTextEditLogger(logging.Handler):
def __init__(self, parent):
super().__init__()
self.widget = QtWidgets.QPlainTextEdit(parent)
self.widget.setReadOnly(True)
def emit(self, record):
msg = self.format(record)
self.widget.appendPlainText(msg)
class MyDialog(QtWidgets.QDialog, QtWidgets.QPlainTextEdit):
def __init__(self, parent=None):
super().__init__(parent)
logTextBox = QTextEditLogger(self)
# You can format what is printed to text box
logTextBox.setFormatter(logging.Formatter('%(asctime)s - %(levelname)s - %(message)s'))
logging.getLogger().addHandler(logTextBox)
# You can control the logging level
logging.getLogger().setLevel(logging.DEBUG)
self._button = QtWidgets.QPushButton(self)
self._button.setText('Test Me')
layout = QtWidgets.QVBoxLayout()
# Add the new logging box widget to the layout
layout.addWidget(logTextBox.widget)
layout.addWidget(self._button)
self.setLayout(layout)
# Connect signal to slot
self._button.clicked.connect(self.test)
def test(self):
logging.debug('damn, a bug')
logging.info('something to remember')
logging.warning('that\'s not right')
logging.error('foobar')
app = QtWidgets.QApplication(sys.argv)
dlg = MyDialog()
dlg.show()
dlg.raise_()
sys.exit(app.exec_())
Threading issues
Note: as of late 2022, this is still the highest ranked answer. The OP doesn't seem to be active anymore, so I took the liberty of editing it because the lack of explanation of the implications of its usage is a common cause of closely related questions.
Be aware that the above code will only work for single threaded programs. If you are sure that your program doesn't use threading, then it's fine. Be aware, though: you have to be completely sure about that, also considering external modules. Qt included.
Don't underestimate this aspect: even some Qt classes that are thread safe are actually not single threaded; for instance, QFileSystemModel is a thread safe object, but it does use threading for file system crawling. If it faces a problem for any reason (ie. access permissions), it will output some debug messages from those threads, and you'll probably still get issues.
Possible results of underestimated threading issues while accessing UI elements include:
unexpected behavior;
drawing artifacts;
lots of "unrelated" debug messages;
possible fatal crash of the program;
So, as a rule of thumb, the above has to be considered as unsafe and should not be used, since it attempts to access the QPlainTextEdit widget even from external threads: widgets are not thread-safe, and can only be accessed from the main thread.
Since there is no immediate way to know if the logging source is in the same thread or not without pointlessly affecting performance, it's better to always use a thread safe solution in any case (see the other answers that consider this aspect).
If you are using the Python logging module to can easily create a custom logging handler that passes the log messages through to a QPlainTextEdit instance (as described by Christopher).
To do this you first subclass logging.Handler. In this __init__ we create the QPlainTextEdit that will contain the logs. The key bit here is that the handle will be receiving messages via the emit() function. So we overload this function and pass the message text into the QPlainTextEdit.
import logging
class QPlainTextEditLogger(logging.Handler):
def __init__(self, parent):
super(QPlainTextEditLogger, self).__init__()
self.widget = QPlainTextEdit(parent)
self.widget.setReadOnly(True)
def emit(self, record):
msg = self.format(record)
self.widget.appendPlainText(msg)
def write(self, m):
pass
Create an object from this class, passing it the parent for the QPlainTextEdit (e.g. the main window, or a layout). You can then add this handler for the current logger.
# Set up logging to use your widget as a handler
log_handler = QPlainTextEditLogger(<parent widget>)
logging.getLogger().addHandler(log_handler)
Here's a complete working example based on mfitzp's answer:
import sys
from PyQt4 import QtCore, QtGui
import logging
# Uncomment below for terminal log messages
# logging.basicConfig(level=logging.DEBUG, format=' %(asctime)s - %(name)s - %(levelname)s - %(message)s')
class QPlainTextEditLogger(logging.Handler):
def __init__(self, parent):
super().__init__()
self.widget = QtGui.QPlainTextEdit(parent)
self.widget.setReadOnly(True)
def emit(self, record):
msg = self.format(record)
self.widget.appendPlainText(msg)
class MyDialog(QtGui.QDialog, QPlainTextEditLogger):
def __init__(self, parent=None):
super().__init__(parent)
logTextBox = QPlainTextEditLogger(self)
# You can format what is printed to text box
logTextBox.setFormatter(logging.Formatter('%(asctime)s - %(levelname)s - %(message)s'))
logging.getLogger().addHandler(logTextBox)
# You can control the logging level
logging.getLogger().setLevel(logging.DEBUG)
self._button = QtGui.QPushButton(self)
self._button.setText('Test Me')
layout = QtGui.QVBoxLayout()
# Add the new logging box widget to the layout
layout.addWidget(logTextBox.widget)
layout.addWidget(self._button)
self.setLayout(layout)
# Connect signal to slot
self._button.clicked.connect(self.test)
def test(self):
logging.debug('damn, a bug')
logging.info('something to remember')
logging.warning('that\'s not right')
logging.error('foobar')
if (__name__ == '__main__'):
app = None
if (not QtGui.QApplication.instance()):
app = QtGui.QApplication([])
dlg = MyDialog()
dlg.show()
dlg.raise_()
if (app):
app.exec_()
Thread-safe version
class QTextEditLogger(logging.Handler, QtCore.QObject):
appendPlainText = QtCore.pyqtSignal(str)
def __init__(self, parent):
super().__init__()
QtCore.QObject.__init__(self)
self.widget = QtWidgets.QPlainTextEdit(parent)
self.widget.setReadOnly(True)
self.appendPlainText.connect(self.widget.appendPlainText)
def emit(self, record):
msg = self.format(record)
self.appendPlainText.emit(msg)
Usage
logTextBox = QTextEditLogger(self)
# log to text box
logTextBox.setFormatter(
logging.Formatter(
'%(asctime)s %(levelname)s %(module)s %(funcName)s %(message)s'))
logging.getLogger().addHandler(logTextBox)
logging.getLogger().setLevel(logging.DEBUG)
# log to file
fh = logging.FileHandler('my-log.log')
fh.setLevel(logging.DEBUG)
fh.setFormatter(
logging.Formatter(
'%(asctime)s %(levelname)s %(module)s %(funcName)s %(message)s'))
logging.getLogger().addHandler(fh)
Alex's answer should be ok in a single thread scenario, but if you are logging in another thread (QThread) you may get the following warning:
QObject::connect: Cannot queue arguments of type 'QTextCursor'
(Make sure 'QTextCursor' is registered using qRegisterMetaType().)
This is because you are modifying the GUI (self.widget.appendPlainText(msg)) from a thread other than the main thread without using the Qt Signal/Slot mechanism.
Here is my solution:
# my_logger.py
import logging
from PyQt5.QtCore import pyqtSignal, QObject
class Handler(QObject, logging.Handler):
new_record = pyqtSignal(object)
def __init__(self, parent):
super().__init__(parent)
super(logging.Handler).__init__()
formatter = Formatter('%(asctime)s|%(levelname)s|%(message)s|', '%d/%m/%Y %H:%M:%S')
self.setFormatter(formatter)
def emit(self, record):
msg = self.format(record)
self.new_record.emit(msg) # <---- emit signal here
class Formatter(logging.Formatter):
def formatException(self, ei):
result = super(Formatter, self).formatException(ei)
return result
def format(self, record):
s = super(Formatter, self).format(record)
if record.exc_text:
s = s.replace('\n', '')
return s
# gui.py
... # GUI code
...
def setup_logger(self)
handler = Handler(self)
log_text_box = QPlainTextEdit(self)
self.main_layout.addWidget(log_text_box)
logging.getLogger().addHandler(handler)
logging.getLogger().setLevel(logging.INFO)
handler.new_record.connect(log_text_box.appendPlainText) # <---- connect QPlainTextEdit.appendPlainText slot
...
Sounds like you'll want to use a QPlainTextEdit widget set to read-only.
Consider changing the background color to gray to give the user a hint that it is not editable. It is also up to you if you want it to be scrollable or the text selectable.
This answer can get you started subclassing QPlainTextEdit to scroll with output, save to a file, whatever.
Sifferman’s answer appears to be the most elegant to me. Regardless, I've tried all in this post. They all work, but notice that if you try, e.g., to write a test and create a log entry on it you might get a nasty error like
RuntimeError: wrapped C/C++ object of type QPlainTextEdit has been deleted
After loosing my mind for a couple hours, I noticed that it's quite important to delete the handler manually when closing the window,
def closeEvent(self, event):
...
root_logger = logging.getLogger()
root_logger.removeHandler(self.logger)
...
super().closeEvent(event)
I am very new to this PyQt, so please I need some help. I created a MainWindow which is imported from another file. collect_host_status also imported from another py file. Basicly the GUI works, but freezing obviously, so therefore I need to use threading for long running process. So far I have changed my code to be like this, but when I click on the button which suppose to check the hosts, happens nothing. :( I dont really get it how to connect the textEdit from MainWindow class to Worker class. As how it is now, it seems like Worker class has no clue what is really self.ui.textEdit.
class Worker(QObject):
finished = Signal()
def __init__(self):
super(Worker, self).__init__()
def run(self):
hostname = self.ui.textEdit.toPlainText()
output_text = collect_host_status(hostname)
for i in output_text:
if "not found" in i:
w = i.replace(" not found", "")
self.ui.textEdit_3.append(w)
else:
self.ui.textEdit_2.append(i)
self.finished.emit()
class MainWindow(QMainWindow):
def __init__(self):
QMainWindow.__init__(self)
self.ui = Ui_MainWindow()
self.ui.setupUi(self)
self.ui.exitbutton.clicked.connect(self.close)
self.ui.actionExit_2.triggered.connect(self.close)
self.ui.actionOpen_2.triggered.connect(self.openfiles)
self.ui.pushButton.clicked.connect(self.ui.textEdit.clear)
self.ui.pushButton.clicked.connect(self.ui.textEdit_2.clear)
self.ui.pushButton.clicked.connect(self.ui.textEdit_3.clear)
self.ui.pushButton_2.clicked.connect(self.ui.textEdit_2.clear)
self.ui.pushButton_2.clicked.connect(self.ui.textEdit_3.clear)
self.connect(self.ui.pushButton_2, SIGNAL("clicked()",), self.buttonclicked)
def buttonclicked(self):
self.thread = QThread()
self.worker = Worker()
self.worker.moveToThread(self.thread)
self.thread.started.connect(self.worker.run)
self.worker.finished.connect(self.thread.quit)
self.worker.finished.connect(self.worker.deleteLater)
self.thread.finished.connect(self.thread.deleteLater)
self.thread.start()
Qt, as with most UI frameworks, does not allow any kind of access to objects from outside its main thread, which means that external threads cannot create widgets, reading properties (like toPlainText()) is unreliable and writing (such as using append()) might even lead to crash.
Even assuming that that was possible, as you pointed out the worker has no clue about self.ui.textEdit and other objects, and that's pretty obvious: self.ui is an attribute created on the main window instance, the thread has no ui object. I suggest you to do some research about how classes and instances work, and how their attributes are accessible.
The only safe and correct way to do so is to use custom signals that are emitted from the thread and connected to the slots (functions) that will actually manipulate the UI.
In the following code I made some adjustments to make it working:
the worker thread directly subclasses from QThread;
only one worker thread is created, and it uses a Queue to get requests from the main thread;
two specialized signals are used to notify whether the request is valid or not, and directly connected to the append() function of the QTextEdits;
I removed the finished signal, which is unnecessary since the thread is going to be reused (and, in any case, QThread already provides such a signal);
changed the "old style" self.connect as it's considered obsolete and will not be supported in newer versions of Qt;
class Worker(QThread):
found = Signal(str)
notFound = Signal(str)
def __init__(self):
QThread.__init__(self)
self.queue = Queue()
def run(self):
while True:
hostname = self.queue.get()
output_text = collect_host_status(hostname)
for i in output_text:
if "not found" in i:
self.notFound.emit(i.replace(" not found", ""))
else:
self.found.emit(i)
def lookUp(self, hostname):
self.queue.put(hostname)
class MainWindow(QMainWindow):
def __init__(self):
# ...
self.ui.pushButton_2.clicked.connect(self.buttonclicked)
self.thread = Worker()
self.thread.found.connect(self.ui.textEdit_2.append)
self.thread.notFound.connect(self.ui.textEdit_3.append)
self.thread.start()
def buttonclicked(self):
if self.ui.textEdit.toPlainText():
self.thread.lookUp(self.ui.textEdit.toPlainText())
What does a Ruby logger that looks like this do?
logger = Logger.new('foo.log', 'w')
The closest thing I can find on the docs is
logger = Logger.new('foo.log', 'weekly')
Or perhaps write only?
file = File.open('foo.log', File::WRONLY | File::APPEND)
I'd assume it means weekly?
TL;DR - The 'w' itself means nothing when creating a new Logger object. But 'weekly' is a viable argument.
I am not sure where you got your initial example from. However, based on the docs, the second (optional) argument of Logger::new is shift_age:
Number of old log files to keep, or frequency of rotation (daily, weekly or monthly).
Any option other than those above will still create a successful log object. (Tested in 1.9.3)
[1] pry(main)> logger = Logger.new('foo.log', 'nothing')
=> #<Logger:0x00000001023590
#default_formatter=#<Logger::Formatter:0x00000001023540 #datetime_format=nil>,
#formatter=nil,
#level=0,
#logdev=
#<Logger::LogDevice:0x000000010234a0
#dev=#<File:foo.log>,
#filename="foo.log",
#mutex=#<Logger::LogDevice::LogDeviceMutex:0x00000001023450 #mon_count=0, #mon_mutex=#<Mutex:0x00000001023388>, #mon_owner=nil>,
#shift_age="nothing",
#shift_size=1048576>,
#progname=nil>
Knowing this, the 'w' itself does not mean write-only when creating a new logger. Just because a logger may write to a file, the logger and file are still two different objects.
More information on HOW TO create a logger
I have the basic PBS Controller and EngineSet Launchers working with my PBS/torque cluster. But now I'd like to pass in additional options through the config file and/or the command line to be rendered in the batch_template file using the {} template formatter syntax.
As an example, I want to run:
ipcluster start -n 10 --pmem=10gb
And then have, in my template:
#PBS -l pmem={pmem}
I guess that I have to subclass the PBSControllerLauncher & PBSEngineSetLauncher classes? This is mentioned in the documentation, but I can't find a clear example of how to do it.
Simply inheriting from these classes and adding pmem = Unicode(...) as a class attribute doesn't make pmem available in the template's context when it calls write_batch_script.
I can access the config options and insert it into the context, as shown below. This code works, but is pretty awkward and doesn't seem like the right way to do it:
from IPython.parallel.apps.launcher import PBSControllerLauncher, PBSEngineSetLauncher, BatchClusterAppMixin, ipcontroller_cmd_argv, ipengine_cmd_argv
from IPython.utils.traitlets import (
Any, Integer, CFloat, List, Unicode, Dict, Instance, HasTraits, CRegExp
)
import pipes
class MyPBSLauncherExtension(object):
def update_context(self):
cfg = self.config
ctx = self.context
cls = type(self).__name__
for c in ('extra_pbs_directives', 'extra_modules'):
if cls in cfg:
if c in cfg[cls]:
ctx[c] = cfg[cls][c]
continue
if c in cfg:
ctx[c] = cfg[c]
else:
ctx[c] = getattr(self, c)
class MyPBSControllerLauncher(PBSControllerLauncher, MyPBSLauncherExtension):
extra_pbs_directives = Unicode(u'', config=True, help='')
extra_modules = Unicode(u'', config=True, help='')
def write_batch_script(self, n):
self.update_context()
super(MyPBSControllerLauncher, self).write_batch_script(n)
class MyPBSEngineSetLauncher(PBSEngineSetLauncher, MyPBSLauncherExtension):
extra_pbs_directives = Unicode(u'', config=True, help='')
extra_modules = Unicode(u'', config=True, help='')
def write_batch_script(self, n):
self.update_context()
super(MyPBSEngineSetLauncher, self).write_batch_script(n)
Would very much appreciate a simple example that adds an additional option to a launcher subclass.
I need to create a custom log file within Padrino that contains all of the logging information that is in stdout as well as custom log messages. I have been able to get the custom log file created, but the stdout file (development.log, production.log, etc.) still gets created with logging statements in it. I have tried putting these lines in the boot.rb file, but none of these seem to work:
Padrino::Logger::Config[:development][:stream] = :to_file
Padrino::Logger::Config[:development] = { :log_level => :debug, :stream => :to_file }
Padrino::Logger::Config[:development][:stream] = :null
Padrino::Logger::Config[:development] = { :log_level => :debug, :stream => :null}
I have looked at Padrino's development commands and logger documentation but they didn't help.
In case it helps, this is the code that is generating the custom log file. (Whether I run this code or not, the stdout file keeps getting created):
log_path = File.join(custom_log_path, 'My Service')
FileUtils.mkdir_p log_path
log_file_path = File.join(log_path, "MyService_#{current_date_time_formatted}.log")
logger = File.open(log_file_path, "a+")
if defined?(PADRINO_ENV) && PADRINO_ENV == 'production'
$stdout.reopen(logger)
$stderr.reopen(logger)
end
Any help is greatly appreciated!
You should be able to do this:
Padrino::Logger::Config[:development][:stream] = logger
# or
Padrino::Logger::Config[:production][:stream] = logger
after you have defined logger. If config[:stream] doesn't receive a keyword, Padrino::Logger will use whatever is passed as the output stream.
For more information on the Padrino logger, check out the relevant Padrino Core code, especially self.setup!: https://github.com/padrino/padrino-framework/blob/master/padrino-core/lib/padrino-core/logger.rb.