I have some data which are represented by a class of my own ; to fix the ideas I give an example.
class MyOwnModel():
def __init__(self, name="", number=0):
self.name = name
self.number = number
I then have a list of such instances, that I want to represent in a QTableView.
li = [MyOwnModel("a", 1), MyOwnModel("b", 2)]
Then I see two strategies to make a QTableView from that :
change MyOwnModel so that it subclasses QAbstractTableModel
build a new QAbstractTableModel which mimics MyOwnModel in a way that its attributes are for instance two QString and connect the dataChanged signal to a function which updates the instance of MyOwnModel
I am not completely satisfied with any of these, but I have no other idea for the moment.
Which one is the most suitable to my problem ? (I have a more complex class in practice but I would like to use the same framework)
As stated in the comment, your model is your list of object. You should subclass QAbstractTableModel to use this list.
Here's my code snippet for this:
import sys
import signal
import PyQt4.QtCore as PCore
import PyQt4.QtGui as PGui
class OneRow(PCore.QObject):
def __init__(self):
self.column0="text in column 0"
self.column1="text in column 1"
class TableModel(PCore.QAbstractTableModel):
def __init__(self):
super(TableModel,self).__init__()
self.myList=[]
def addRow(self,rowObject):
row=len(self.myList)
self.beginInsertRows(PCore.QModelIndex(),row,row)
self.myList.append(rowObject)
self.endInsertRows()
#number of row
def rowCount(self,QModelIndex):
return len(self.myList)
#number of columns
def columnCount(self,QModelIndex):
return 2
#Define what do you print in the cells
def data(self,index,role):
row=index.row()
col=index.column()
if role==PCore.Qt.DisplayRole:
if col==0:
return str( self.myList[row].column0)
if col==1:
return str( self.myList[row].column1)
#Rename the columns
def headerData(self,section,orientation,role):
if role==PCore.Qt.DisplayRole:
if orientation==PCore.Qt.Horizontal:
if section==0:
return str("Column 1")
elif section==1:
return str("Column 2")
if __name__=='__main__':
PGui.QApplication.setStyle("plastique")
app=PGui.QApplication(sys.argv)
#Model
model=TableModel()
model.addRow(OneRow())
model.addRow(OneRow())
#View
win=PGui.QTableView()
win.setModel(model)
#to be able to close wth ctrl+c
signal.signal(signal.SIGINT, signal.SIG_DFL)
#to avoid warning when closing
win.setAttribute(PCore.Qt.WA_DeleteOnClose)
win.show()
sys.exit(app.exec_())
Each element of myList is a row in the table.
Related
I want to recreate the training dataset by a function generate_custom_train_set at the beginning of every epoch, however, is there a way I could do it with Trainer using callback?
My trainer looks like
trainer = Trainer(
model=model,
args=args,
train_dataset=train_dataset.,
eval_dataset=validation_dataset,
tokenizer=tokenizer,
)
I'm having the same question as I try to implement Examples-proportional mixing from the T5 paper. I didn't find support from hugging face.
My current solution is to modify the trainer.train_dataset in the on_epoch_begin callback.
Here's an implementation. I'm using this in my own project. Seems to work.
First, implement your per-epoch change in your Dataset, in my case, it's the sample function for Examples-Proportional Mixing.
class ProportionMixingDataset:
"""
Examples-proportional mixing from T5
TODO: failed to find a pytorch working implementation
Equivalent to, for the larger datasets, a new subset is taken at each epoch,
then sample in the joined subset once
"""
def __init__(self, dataset_list: List[Dataset] = None, k: int = None):
"""
:param dataset_list: Ordered list of datasets
:param k: Artificial limit
"""
self.dsets = dataset_list
assert k is not None
self.k = k
self.dset_szs = [min(len(d), k) for d in self.dsets]
self.sz = sum(self.dset_szs)
self._sampled_idxs: List[Optional[torch.Tensor]] = [None] * len(self.dsets)
self.sample()
def sample(self):
"""
Sub-sample datasets larger than k
Intended to call in each epoch
"""
for i, dset in enumerate(self.dsets):
sz = len(dset)
if sz > self.k:
self._sampled_idxs[i] = torch.randperm(sz)[:self.k]
def __len__(self):
return self.sz
def _idx2dset_idx(self, idx: int) -> Tuple[int, int]:
"""
Convert a global index to a dataset index
"""
for i, sz in enumerate(self.dset_szs):
if idx < sz:
return i, idx
idx -= sz
raise ValueError('Should not happen')
def __getitem__(self, idx):
if not isinstance(idx, int):
raise ValueError('Batched indexing not supported')
idx_dset, idx = self._idx2dset_idx(idx)
dset = self.dsets[idx_dset]
if self._sampled_idxs[idx_dset] is not None: # A sub-sample index
idx = self._sampled_idxs[idx_dset][idx].item()
return dset[idx]
Then pass that dataset to Trainer.
Now comes the magic part:
class ProportionalMixCallback(TrainerCallback):
"""
Trigger re-computing subset for dataset Examples-proportional mixing, see `dataset::ProportionMixingDataset`
A hack that modifies the train dataset, pointed by Trainer's dataloader
"""
def __init__(self, trainer: Trainer):
self.trainer = trainer
def on_epoch_begin(self, args: TrainingArguments, state, control, **kwargs):
self.trainer.train_dataset.sample()
Pass this to your trainer as a callback.
This triggers the sample call which modifies the dataset at the times we need it.
This works becasue train_dataLoader in trainer still points to the same train dataset object.
I need to use a custom QtableView in python to display and format data.
The example app below shows a table with in the first column floats formatted as strings to get proper number of decimals, second column are pure float displayed so without formatting and the third one are strings.
When clicking on columns I want to sort my data which works fine for strings and floats (columns #2 and #3) but not for my column #1 with formatted floats as strings where it's sorted alphabetically rather than numerically.
I'm googling since a while without finding a way to have something working with QtableView.
Any clue on how to get both floats sorting and decimal formatting ?
Thanks & cheers
Stephane
import sys
from PyQt5 import QtCore, QtWidgets
from PyQt5.QtCore import *
# Table model
class TableModel(QtCore.QAbstractTableModel):
def __init__(self, data):
super(TableModel, self).__init__()
self._data = data
# Set columns headers
self.horizontalHeaders = [''] * 3
self.setHeaderData(0, Qt.Horizontal, "Col #1\nfloats as string")
self.setHeaderData(1, Qt.Horizontal, "Col #2\nfloats")
self.setHeaderData(2, Qt.Horizontal, "Col #3\nstrings")
def data(self, index, role):
value = self._data[index.row()][index.column()]
if role == Qt.DisplayRole:
# convert col #1 from floats to string to get proper number of decimal formatting
if index.column() == 0:
return '%.4f' % value
# otherwise display floats or strings for col #2 and #3
else:
return value
# Align values right
if role == Qt.TextAlignmentRole:
return Qt.AlignVCenter + Qt.AlignRight
def rowCount(self, index):
# The length of the outer list.
return len(self._data)
def columnCount(self, index):
# The following takes the first sub-list, and returns
# the length (only works if all rows are an equal length)
return len(self._data[0])
def setHeaderData(self, section, orientation, data, role=Qt.EditRole):
if orientation == Qt.Horizontal and role in (Qt.DisplayRole, Qt.EditRole):
try:
self.horizontalHeaders[section] = data
return True
except:
return False
return super().setHeaderData(section, orientation, data, role)
def headerData(self, section, orientation, role=Qt.DisplayRole):
if orientation == Qt.Horizontal and role == Qt.DisplayRole:
try:
return self.horizontalHeaders[section]
except:
pass
return super().headerData(section, orientation, role)
class MainWindow(QtWidgets.QMainWindow):
def __init__(self):
super().__init__()
# Create a TableView (not a TableWidget !!!)
self.table = QtWidgets.QTableView()
# sample data
data = [
[4.2, 9.6, 1],
[42.1, 0.0, 11],
[3.1, 5.55, 2],
[30.0, 3.55, 2222],
[7.99, 8.99, 33],
]
# Set table model
self.model = TableModel(data)
self.table.setModel(self.model)
self.setCentralWidget(self.table)
# Use proxy for column sorting
proxyModel = QSortFilterProxyModel()
proxyModel.setSourceModel(self.model)
self.table.setModel(proxyModel)
self.table.setSortingEnabled(True)
# hide vertical headers
self.table.verticalHeader().setVisible(False)
# format horizontal headers
stylesheet = "::section{Background-color:rgb(171,178,185);font-weight:bold}"
self.table.setStyleSheet(stylesheet)
self.table.setAlternatingRowColors(True)
if __name__ == '__main__':
app = QtWidgets.QApplication(sys.argv)
window = MainWindow()
window.setMinimumSize(350, 250)
window.setWindowTitle('Sorting column example')
window.show()
app.exec_()
Thanks to a colleague I've found an implementation which works.
Basically one has to override the sorting function and not using the QSortFilterProxyModel() function but rewrite your own function
this new sorting function will be called and just do a custom sorting
Here is the modified code which now works fine for any type of data.
import sys
from PyQt5 import QtCore, QtWidgets
from PyQt5.QtCore import *
# Table model
class TableModel(QtCore.QAbstractTableModel):
def __init__(self, data):
super(TableModel, self).__init__()
self._data = data
# Set columns headers
self.horizontalHeaders = [''] * 3
self.setHeaderData(0, Qt.Horizontal, "Col #1\nfloats as string")
self.setHeaderData(1, Qt.Horizontal, "Col #2\nfloats")
self.setHeaderData(2, Qt.Horizontal, "Col #3\nstrings")
def data(self, index, role):
value = self._data[index.row()][index.column()]
if role == Qt.DisplayRole:
# convert col #1 from floats to string to get proper number of decimal formatting
if index.column() == 0:
return '%.4f' % value
# otherwise display floats or strings for col #2 and #3
else:
return value
if role == Qt.UserRole:
return value
# Align values right
if role == Qt.TextAlignmentRole:
return Qt.AlignVCenter + Qt.AlignRight
def rowCount(self, index):
# The length of the outer list.
return len(self._data)
def columnCount(self, index):
# The following takes the first sub-list, and returns
# the length (only works if all rows are an equal length)
return len(self._data[0])
def setHeaderData(self, section, orientation, data, role=Qt.EditRole):
if orientation == Qt.Horizontal and role in (Qt.DisplayRole, Qt.EditRole):
try:
self.horizontalHeaders[section] = data
return True
except:
return False
return super().setHeaderData(section, orientation, data, role)
def headerData(self, section, orientation, role=Qt.DisplayRole):
if orientation == Qt.Horizontal and role == Qt.DisplayRole:
try:
return self.horizontalHeaders[section]
except:
pass
return super().headerData(section, orientation, role)
class mysortingproxy(QSortFilterProxyModel):
def __init__(self):
super(mysortingproxy, self).__init__()
def lessThan(self, left: QModelIndex, right: QModelIndex) -> bool:
leftDqtq = self.sourceModel().data(left, Qt.UserRole)
rightDqtq = self.sourceModel().data(right, Qt.UserRole)
return leftDqtq < rightDqtq
class MainWindow(QtWidgets.QMainWindow):
def __init__(self):
super().__init__()
# Create a TableView (not a TableWidget !!!)
self.table = QtWidgets.QTableView()
# sample data
data = [
[4.2, 9.6, 1],
[42.1, 0.0, 11],
[3.1, 5.55, 2],
[30.0, 3.55, 2222],
[7.99, 8.99, 33],
]
# Set table model
self.model = TableModel(data)
self.table.setModel(self.model)
self.setCentralWidget(self.table)
# Use proxy for column sorting overriding the QSortFilterProxyModel() function with a custom sorting proxy function
proxyModel = mysortingproxy()
proxyModel.setSourceModel(self.model)
self.table.setModel(proxyModel)
self.table.setSortingEnabled(True)
# hide vertical headers
self.table.verticalHeader().setVisible(False)
# format horizontal headers
stylesheet = "::section{Background-color:rgb(171,178,185);font-weight:bold}"
self.table.setStyleSheet(stylesheet)
self.table.setAlternatingRowColors(True)
if __name__ == '__main__':
app = QtWidgets.QApplication(sys.argv)
window = MainWindow()
window.setMinimumSize(350, 250)
window.setWindowTitle('Sorting column example')
window.show()
app.exec_()
I want to load a YAML file into Python as an OrderedDict. I am using yamlordereddictloader to preserve ordering.
However, I notice that the aliased object is placed "too soon" in the OrderedDict in the output.
How can I preserve the order of this mapping when read into Python, ideally as an OrderedDict? Is it possible to achieve this result without writing some custom parsing?
Notes:
I'm not particularly concerned with the method used, as long as the end result is the same.
Using sequences instead of mappings is problematic because they can result in nested output, and I can't simply flatten everything (some nestedness is appropriate).
When I try to just use !!omap, I cannot seem to merge the aliased mapping (d1.dt) into the d2 mapping.
I'm in Python 3.6, if I don't use this loader or !!omap order is not preserved (apparently contrary to the top 'Update' here: https://stackoverflow.com/a/21912744/2343633)
import yaml
import yamlordereddictloader
yaml_file = """
d1:
id:
nm1: val1
dt: &dt
nm2: val2
nm3: val3
d2: # expect nm4, nm2, nm3
nm4: val4
<<: *dt
"""
out = yaml.load(yaml_file, Loader=yamlordereddictloader.Loader)
keys = [x for x in out['d2']]
print(keys) # ['nm2', 'nm3', 'nm4']
assert keys==['nm4', 'nm2', 'nm3'], "order from YAML file is not preserved, aliased keys placed too early"
Is it possible to achieve this result without writing some custom parsing?
Yes. You need to override the method flatten_mapping from SafeConstructor. Here's a basic working example:
import yaml
import yamlordereddictloader
from yaml.constructor import *
from yaml.reader import *
from yaml.parser import *
from yaml.resolver import *
from yaml.composer import *
from yaml.scanner import *
from yaml.nodes import *
class MyLoader(yamlordereddictloader.Loader):
def __init__(self, stream):
yamlordereddictloader.Loader.__init__(self, stream)
# taken from here and reengineered to keep order:
# https://github.com/yaml/pyyaml/blob/5.3.1/lib/yaml/constructor.py#L207
def flatten_mapping(self, node):
merged = []
def merge_from(node):
if not isinstance(node, MappingNode):
raise yaml.ConstructorError("while constructing a mapping",
node.start_mark, "expected mapping for merging, but found %s" %
node.id, node.start_mark)
self.flatten_mapping(node)
merged.extend(node.value)
for index in range(len(node.value)):
key_node, value_node = node.value[index]
if key_node.tag == u'tag:yaml.org,2002:merge':
if isinstance(value_node, SequenceNode):
for subnode in value_node.value:
merge_from(subnode)
else:
merge_from(value_node)
else:
if key_node.tag == u'tag:yaml.org,2002:value':
key_node.tag = u'tag:yaml.org,2002:str'
merged.append((key_node, value_node))
node.value = merged
yaml_file = """
d1:
id:
nm1: val1
dt: &dt
nm2: val2
nm3: val3
d2: # expect nm4, nm2, nm3
nm4: val4
<<: *dt
"""
out = yaml.load(yaml_file, Loader=MyLoader)
keys = [x for x in out['d2']]
print(keys)
assert keys==['nm4', 'nm2', 'nm3'], "order from YAML file is not preserved, aliased keys placed too early"
This has not the best performance as it basically copies all key-value pairs from all mappings once each during loading, but it's working. Performance enhancement is left as an exercise for the reader :).
I have a simple GtkTreeView and a GtkEntry used to filter the model.
When I type somehing into the entry, software_list is filtered by language.
software_list = [("Firefox", 2002, "C++"),
("Eclipse", 2004, "Java" ),
("Netbeans", 1996, "Java"),
("Chrome", 2008, "C++"),
("GCC", 1987, "C"),
("Frostwire", 2004, "Java")]
class TreeViewFilterWindow(Gtk.Window):
def __init__(self):
Gtk.Window.__init__(self)
self.curr_filter = ''
self.entry = Gtk.Entry()
self.entry.connect('changed', self.on_text_change)
self.software_liststore = Gtk.ListStore(str, int, str)
for software_ref in software_list:
self.software_liststore.append(list(software_ref))
self.filter = self.software_liststore.filter_new()
self.filter.set_visible_func(self.filter_func)
self.treeview = Gtk.TreeView.new_with_model(self.filter)
for i, column_title in enumerate(["Software", "Release Year", "Programming Language"]):
renderer = Gtk.CellRendererText()
column = Gtk.TreeViewColumn(column_title, renderer, text=i)
self.treeview.append_column(column)
self.treeview.get_selection().connect('changed', self.on_row_select)
# packing into boxes, showing components, starting main loop goes here
def on_text_change(self, entry):
self.curr_filter = entry.get_text()
self.filter.refilter()
def filter_func(self, model, iter, data):
if self.curr_filter:
return re.search(re.escape(self.curr_filter), model[iter][2])
else:
return True
The problem is, when I select i.e. "Chrome" from the list and then type "Java" into the entry, then, obviously, "Chrome" gets hidden but selection changes to some other, random row. I'd prefer TreeView unselected hidden elements instead of changing the selection. How can I do this?
This just works as expected in Gtk2, but in Gtk3 you need to deselect the row if it disappears. The appropriate code is
class TreeViewFilterWindow(Gtk.Window):
def __init__(...):
...
self.selection = self.treeview.get_selection()
self.filter.connect('row-deleted', self.on_row_deleted)
def on_row_deleted(self, model, path):
if self.selection.path_is_selected(path):
GObject.idle_add(self.selection.unselect_path, path)
I found that calling self.selection.unselect_path(path) directly didn't seem to work for some reason, but deferring it with idle_add sorted it out.
When you print out a QModelIndex in Pyside, the object representation shows the row, column, parent, model, and memory address. However, if you print out index.model(), the memory address for the model is different.
Here is some code that demonstrates what I mean:
from PySide import QtGui, QtCore
class TestQModelIndexModelWin(QtGui.QMainWindow):
def __init__(self, parent=None):
super(TestQModelIndexModelWin, self).__init__(parent)
self.listView = QtGui.QListView()
self.setCentralWidget(self.listView)
listModel = QtGui.QStringListModel(['foo', 'bar', 'baz'])
self.listView.setModel(listModel)
numItems = len(listModel.stringList())
for i in range(numItems):
index = listModel.index(i, 0)
print index
print index.model()
When running this code, the results look something like the following:
<PySide.QtCore.QModelIndex(0,0,0x0,QStringListModel(0xef1b7e0) ) at 0x0000000017656D08>
<PySide.QtGui.QStringListModel object at 0x0000000017656948>
<PySide.QtCore.QModelIndex(1,0,0x0,QStringListModel(0xef1b7e0) ) at 0x00000000176564C8>
<PySide.QtGui.QStringListModel object at 0x0000000017656948>
<PySide.QtCore.QModelIndex(2,0,0x0,QStringListModel(0xef1b7e0) ) at 0x0000000017656D08>
<PySide.QtGui.QStringListModel object at 0x0000000017656948>
Why does the QModelIndex show the QStringListModel hex value as 0xef1b7e0 but the QStringListModel shows its address as 0x0000000017656948?
The repr for index is showing the C++ address of the model it is associated with. Whereas the repr for index.model() is showing the address of the python object that wraps the C++ model.
You can verify this by using the shiboken module:
import shiboken
...
print index
print index.model()
print shiboken.dump(index.model())
which will produce output like this:
<PySide.QtCore.QModelIndex(2,0,0x0,QStringListModel(0x17b0b40) ) at 0x7ff1a3715998>
<PySide.QtGui.QStringListModel object at 0x7ff1a3715950>
C++ address....... PySide.QtGui.QStringListModel/0x17b0b40
hasOwnership...... 1
containsCppWrapper 1
validCppObject.... 1
wasCreatedByPython 1