How can I export layered drawings from drawio to create "animated" slides in beamer? - animation

When preparing lectures, or conference presentations with beamer, I usually use layered drawings. Then for graphics included in consecutive slides ("frames" in beamer), I simply use different sets of layers.
For graphics created in IPE, I have created a dedicated expallviews.lua script.
Unfortunately, for graphics created with diagrams.net locally run as drawio-desktop, no such automated export of various layers exists. The only way is to manually select the visible layers in GUI and then export consecutive drawings to a set of PDF files.
Is there a more convenient method to solve that problem?

The described problem has been reported in issues 405 and 737 in the drawio-desktop repository.
After reviewing those issues, I have found a method based on automated (instead of a manual via GUI) changing the visibility of layers and exporting such drawings to the set of PDF files. The proposed method is described in the comment to the issue 405. It uses a simple Python script:
#!/usr/bin/python3
"""
This script modifies the visibility of layers in the XML
file with diagram generated by drawio.
It works around the problem of lack of a possibility to export
only the selected layers from the CLI version of drawio.
Written by Wojciech M. Zabolotny 6.10.2022
(wzab01<at>gmail.com or wojciech.zabolotny<at>pw.edu.pl)
The code is published under LGPL V2 license
"""
from lxml import etree as let
import xml.etree.ElementTree as et
import xml.parsers.expat as pe
from io import StringIO
import os
import sys
import shutil
import zlib
import argparse
PARSER = argparse.ArgumentParser()
PARSER.add_argument("--layers", help="Selected layers, \"all\", comma separated list of integers or integer ranges like \"0-3,6,7\"", default="all")
PARSER.add_argument("--layer_prefix", help="Layer name prefix", default="Layer_")
PARSER.add_argument("--outfile", help="Output file", default="output.drawio")
PARSER.add_argument("--infile", help="Input file", default="input.drawio")
ARGS = PARSER.parse_args()
INFILENAME = ARGS.infile
OUTFILENAME = ARGS.outfile
# Find all elements with 'value' starting with the layer prefix.
# Return tuples with the element and the rest of 'value' after the prefix.
def find_layers(el_start):
res = []
for el in el_start:
val = el.get('value')
if val is not None:
if val.find(ARGS.layer_prefix) == 0:
# This is a layer element. Add it, and its name
# after the prefix to the list.
res.append((el,val[len(ARGS.layer_prefix):]))
continue
# If it is not a layer element, scan its children
res.extend(find_layers(el))
return res
# Analyse the list of visible layers, and create the list
# of layers that should be visible. Customize this part
# if you want a more sophisticate method for selection
# of layers.
# Now only "all", comma separated list of integers
# or ranges of integers are supported.
def build_visible_list(layers):
if layers == "all":
return layers
res = []
for lay in layers.split(','):
# Is it a range?
s = lay.find("-")
if s > 0:
# This is a range
first = int(lay[:s])
last = int(lay[(s+1):])
res.extend(range(first,last+1))
else:
res.append(int(lay))
return res
def is_visible(layer_tuple,visible_list):
if visible_list == "all":
return True
if int(layer_tuple[1]) in visible_list:
return True
try:
EL_ROOT = et.fromstring(open(INFILENAME,"r").read())
except et.ParseError as perr:
# Handle the parsing error
ROW, COL = perr.position
print(
"Parsing error "
+ str(perr.code)
+ "("
+ pe.ErrorString(perr.code)
+ ") in column "
+ str(COL)
+ " of the line "
+ str(ROW)
+ " of the file "
+ INFILENAME
)
sys.exit(1)
visible_list = build_visible_list(ARGS.layers)
layers = find_layers(EL_ROOT)
for layer_tuple in layers:
if is_visible(layer_tuple,visible_list):
print("set "+layer_tuple[1]+" to visible")
layer_tuple[0].attrib['visible']="1"
else:
print("set "+layer_tuple[1]+" to invisible")
layer_tuple[0].attrib['visible']="0"
# Now write the modified file
t=et.ElementTree(EL_ROOT)
with open(OUTFILENAME, 'w') as f:
t.write(f, encoding='unicode')
The maintained version of that script, together with a demonstration of its use is also available in my github repository.

Related

How to convert yolo annotations to coco format. Json?

I want to convert my labels in yolo format to coco format
I have tried
https://github.com/Taeyoung96/Yolo-to-COCO-format-converter
And
Pylabel
They all have a bugs.
I want to train on detectron 2 but it fails to load the dataset because of the wrong json file.
Thanks everybody
Could you try with this tool (disclaimer: I'm the author)? It is not (yet) a Python package so you need to downloads the repo first. This should ressemble something like:
from ObjectDetectionEval import *
from pathlib import Path
def main() -> None:
path = Path("/path/to/annotations/") # Where the .txt files are
names_file = Path("/path/to/classes.names")
save_file = Path("coco.json")
annotations = AnnotationSet.from_yolo(gts_path).map_labels(names)
# If you need to change the labels
# names = Annotation.parse_names_file(names_file)
# annotations.map_labels(names)
annotations.save_coco(save_file)
if __name__ == "__main__":
main()
If you need more control (coordinate format, images location and extension, etc.) you should use the more generic AnnotationSet.from_txt(). If it does not suit your needs you can easily implement your own parser using AnnotationSet.from_folder().

Is there any good way to rewrite the edgetpu old code by using pycoral api?

I'm a beginner using coral devboard mini.
I want to start a Smart Bird Feeder project.
https://coral.ai/projects/bird-feeder/
I've been trying to execute the code by referring to
I can't run bird_classify.py.
The error is as follows
untimeError: Internal: Unsupported data type in custom op handler: 0Node number 0 (edgetpu-custom-op) failed to prepare.
Originally, the samples in this project seemed to be deprecated, and
The edgetpu requires an old runtimeversion of 13, instead of the current 14.
(tflite is 2.5 ) I have downloaded it directly and re-installed it in
/usr/lib/python3/dist-packagesm
, but I cannot uninstall the new version and cannot match the version.
Is there a better way to do this?
Also, I've decided to give up on running the same environment as the sample, and use the pycoralapi to run the
If there is a good way to rewrite the code to use pycoral, please let me know.
Thanks
#!/usr/bin/python3
"""
Coral Smart Bird Feeder
Uses ClassificationEngine from the EdgeTPU API to analyze animals in
camera frames. Sounds a deterrent if a squirrel is detected.
Users define model, labels file, storage path, deterrent sound, and
optionally can set this to training mode for collecting images for a custom
model.
"""
import argparse
import time
import re
import imp
import logging
import gstreamer
import sys
sys.path.append('/usr/lib/python3/dist-packages/edgetpu')
from edgetpu.classification.engine import ClassificationEngine
from PIL import Image
from playsound import playsound
from pycoral.adapters import classify
from pycoral.adapters import common
from pycoral.utils.dataset import read_label_file
from pycoral.utils.edgetpu import make_interpreter
def save_data(image,results,path,ext='png'):
"""Saves camera frame and model inference results
to user-defined storage directory."""
tag = '%010d' % int(time.monotonic()*1000)
name = '%s/img-%s.%s' %(path,tag,ext)
image.save(name)
print('Frame saved as: %s' %name)
logging.info('Image: %s Results: %s', tag,results)
def load_labels(path):
"""Parses provided label file for use in model inference."""
p = re.compile(r'\s*(\d+)(.+)')
with open(path, 'r', encoding='utf-8') as f:
lines = (p.match(line).groups() for line in f.readlines())
return {int(num): text.strip() for num, text in lines}
def print_results(start_time, last_time, end_time, results):
"""Print results to terminal for debugging."""
inference_rate = ((end_time - start_time) * 1000)
fps = (1.0/(end_time - last_time))
print('\nInference: %.2f ms, FPS: %.2f fps' % (inference_rate, fps))
for label, score in results:
print(' %s, score=%.2f' %(label, score))
def do_training(results,last_results,top_k):
"""Compares current model results to previous results and returns
true if at least one label difference is detected. Used to collect
images for training a custom model."""
new_labels = [label[0] for label in results]
old_labels = [label[0] for label in last_results]
shared_labels = set(new_labels).intersection(old_labels)
if len(shared_labels) < top_k:
print('Difference detected')
return True
def user_selections():
parser = argparse.ArgumentParser()
parser.add_argument('--model', required=True,
help='.tflite model path')
parser.add_argument('--labels', required=True,
help='label file path')
parser.add_argument('--top_k', type=int, default=3,
help='number of classes with highest score to display')
parser.add_argument('--threshold', type=float, default=0.1,
help='class score threshold')
parser.add_argument('--storage', required=True,
help='File path to store images and results')
parser.add_argument('--sound', required=True,
help='File path to deterrent sound')
parser.add_argument('--print', default=False, required=False,
help='Print inference results to terminal')
parser.add_argument('--training', default=False, required=False,
help='Training mode for image collection')
args = parser.parse_args()
return args
def main():
"""Creates camera pipeline, and pushes pipeline through ClassificationEngine
model. Logs results to user-defined storage. Runs either in training mode to
gather images for custom model creation or in deterrent mode that sounds an
'alarm' if a defined label is detected."""
args = user_selections()
print("Loading %s with %s labels."%(args.model, args.labels))
engine = ClassificationEngine(args.model)
labels = load_labels(args.labels)
storage_dir = args.storage
#Initialize logging file
logging.basicConfig(filename='%s/results.log'%storage_dir,
format='%(asctime)s-%(message)s',
level=logging.DEBUG)
last_time = time.monotonic()
last_results = [('label', 0)]
def user_callback(image,svg_canvas):
nonlocal last_time
nonlocal last_results
start_time = time.monotonic()
results = engine.classify_with_image(image, threshold=args.threshold, top_k=args.top_k)
end_time = time.monotonic()
results = [(labels[i], score) for i, score in results]
if args.print:
print_results(start_time,last_time, end_time, results)
if args.training:
if do_training(results,last_results,args.top_k):
save_data(image,results, storage_dir)
else:
#Custom model mode:
#The labels can be modified to detect/deter user-selected items
if results[0][0] !='background':
save_data(image, storage_dir,results)
if 'fox squirrel, eastern fox squirrel, Sciurus niger' in results:
playsound(args.sound)
logging.info('Deterrent sounded')
last_results=results
last_time = end_time
result = gstreamer.run_pipeline(user_callback)
if __name__ == '__main__':
main()
enter code here
I suggest that you follow one of the examples available from the coral examples. There is an example named classify_image.py which uses the edgetpu (tflite) that I found works. After you install the coral examples, you have to drill down through the directory hierarchy. So, in my case, from root it is: /home/pi/ml-projects/coral/pycoral/tensorflow/examples/lite/examples. There are 17 files in that last examples directory. I'm using: numpy 1.19.3, pycoral 2.0.0, scipy 1.7.1, tensorflow 2.4.0, tflite-runtime 2.5.0.post1. I've installed the following edgetpu-runtime: edgetpu_runtime_20201105.zip.

Modifying label_image.py in TensorFlow tutorial to classify multiple images

I have retrained an InceptionV3 model on my own data and am trying to modify the code from the Tensorflow image classification tutorial here https://www.tensorflow.org/tutorials/image_recognition.
I attempted reading in the directory as a list and looping over it but this didn't work:
load_graph(FLAGS.graph)
filelist = os.listdir(FLAGS.image)
for i in filelist:
# load image
image_data = load_image(i)
I just get an error saying that FLAGS hasn't been defined, so I guess FLAGS has to go together with the load_image function? This is the original program:
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import argparse
import sys
import os
import tensorflow as tf
parser = argparse.ArgumentParser()
parser.add_argument(
'--image', required=True, type=str, help='Absolute path to image file.')
parser.add_argument(
'--num_top_predictions',
type=int,
default=5,
help='Display this many predictions.')
parser.add_argument(
'--graph',
required=True,
type=str,
help='Absolute path to graph file (.pb)')
parser.add_argument(
'--labels',
required=True,
type=str,
help='Absolute path to labels file (.txt)')
parser.add_argument(
'--output_layer',
type=str,
default='final_result:0',
help='Name of the result operation')
parser.add_argument(
'--input_layer',
type=str,
default='DecodeJpeg/contents:0',
help='Name of the input operation')
def load_image(filename):
"""Read in the image_data to be classified."""
return tf.gfile.FastGFile(filename, 'rb').read()
def load_labels(filename):
"""Read in labels, one label per line."""
return [line.rstrip() for line in tf.gfile.GFile(filename)]
def load_graph(filename):
"""Unpersists graph from file as default graph."""
with tf.gfile.FastGFile(filename, 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
tf.import_graph_def(graph_def, name='')
def run_graph(image_data, labels, input_layer_name, output_layer_name,
num_top_predictions):
with tf.Session() as sess:
# Feed the image_data as input to the graph.
# predictions will contain a two-dimensional array, where one
# dimension represents the input image count, and the other has
# predictions per class
softmax_tensor = sess.graph.get_tensor_by_name(output_layer_name)
predictions, = sess.run(softmax_tensor, {input_layer_name: image_data})
# Sort to show labels in order of confidence
top_k = predictions.argsort()[-num_top_predictions:][::-1]
for node_id in top_k:
human_string = labels[node_id]
score = predictions[node_id]
print('%s (score = %.5f)' % (human_string, score))
return 0
def main(argv):
"""Runs inference on an image."""
if argv[1:]:
raise ValueError('Unused Command Line Args: %s' % argv[1:])
if not tf.gfile.Exists(FLAGS.image):
tf.logging.fatal('image file does not exist %s', FLAGS.image)
if not tf.gfile.Exists(FLAGS.labels):
tf.logging.fatal('labels file does not exist %s', FLAGS.labels)
if not tf.gfile.Exists(FLAGS.graph):
tf.logging.fatal('graph file does not exist %s', FLAGS.graph)
# load image
image_data = load_image(FLAGS.image)
# load labels
labels = load_labels(FLAGS.labels)
# load graph, which is stored in the default session
load_graph(FLAGS.graph)
run_graph(image_data, labels, FLAGS.input_layer, FLAGS.output_layer,
FLAGS.num_top_predictions)
if __name__ == '__main__':
FLAGS, unparsed = parser.parse_known_args()
tf.app.run(main=main, argv=sys.argv[:1]+unparsed)
Try tf.flags.FLAGS, or at the top, from tf.flags import FLAGS
Try the following,
import os
import tensorflow as tf
# Define this after your imports. This is similar to python argparse except more verbose
FLAGS = tf.app.flags.FLAGS
tf.app.flags.DEFINE_string('image', '/Users/photos',
"""
Define your 'image' folder here
or as an argument to your script
for eg, test.py --image /Users/..
""")
# use listdir to list the images in the target folder
filelist = os.listdir(FLAGS.image)
# now iterate over the objects in the list
for i in filelist:
# load image
image_data = load_image(i)
This should work. Hope it helps.
Thanks for the help given, The FLAGS come from the argparser module and not the TensorFlow flags module, and FLAGS may have to be called from within a function. I eventually solved this by making a separate function so I think that's what is happening:
def get_image_list(path):
return glob.glob(path + '*.jpg')
Then further down calling a loop:
filelist = get_image_list(FLAGS.image)
for i in filelist:
image_data = load_image(i)
run_graph(image_data, labels, FLAGS.input_layer, FLAGS.output_layer,
FLAGS.num_top_predictions)

Script working in Python2 but not in Python 3 (hashlib)

I worked today in a simple script to checksum files in all available hashlib algorithms (md5, sha1.....) I wrote it and debug it with Python2, but when I decided to port it to Python 3 it just won't work. The funny thing is that it works for small files, but not for big files. I thought there was a problem with the way I was buffering the file, but the error message is what makes me think it is something related to the way I am doing the hexdigest (I think) Here is a copy of my entire script, so feel free to copy it, use it and help me figure out what the problem is with it. The error I get when checksuming a 250 MB file is
"'utf-8' codec can't decode byte 0xf3 in position 10: invalid continuation byte"
I google it, but can't find anything that fixes it. Also if you see better ways to optimize it, please let me know. My main goal is to make work 100% in Python 3. Thanks
#!/usr/local/bin/python33
import hashlib
import argparse
def hashFile(algorithm = "md5", filepaths=[], blockSize=4096):
algorithmType = getattr(hashlib, algorithm.lower())() #Default: hashlib.md5()
#Open file and extract data in chunks
for path in filepaths:
try:
with open(path) as f:
while True:
dataChunk = f.read(blockSize)
if not dataChunk:
break
algorithmType.update(dataChunk.encode())
yield algorithmType.hexdigest()
except Exception as e:
print (e)
def main():
#DEFINE ARGUMENTS
parser = argparse.ArgumentParser()
parser.add_argument('filepaths', nargs="+", help='Specified the path of the file(s) to hash')
parser.add_argument('-a', '--algorithm', action='store', dest='algorithm', default="md5",
help='Specifies what algorithm to use ("md5", "sha1", "sha224", "sha384", "sha512")')
arguments = parser.parse_args()
algo = arguments.algorithm
if algo.lower() in ("md5", "sha1", "sha224", "sha384", "sha512"):
Here is the code that works in Python 2, I will just put it in case you want to use it without having to modigy the one above.
#!/usr/bin/python
import hashlib
import argparse
def hashFile(algorithm = "md5", filepaths=[], blockSize=4096):
'''
Hashes a file. In oder to reduce the amount of memory used by the script, it hashes the file in chunks instead of putting
the whole file in memory
'''
algorithmType = hashlib.new(algorithm) #getattr(hashlib, algorithm.lower())() #Default: hashlib.md5()
#Open file and extract data in chunks
for path in filepaths:
try:
with open(path, mode = 'rb') as f:
while True:
dataChunk = f.read(blockSize)
if not dataChunk:
break
algorithmType.update(dataChunk)
yield algorithmType.hexdigest()
except Exception as e:
print e
def main():
#DEFINE ARGUMENTS
parser = argparse.ArgumentParser()
parser.add_argument('filepaths', nargs="+", help='Specified the path of the file(s) to hash')
parser.add_argument('-a', '--algorithm', action='store', dest='algorithm', default="md5",
help='Specifies what algorithm to use ("md5", "sha1", "sha224", "sha384", "sha512")')
arguments = parser.parse_args()
#Call generator function to yield hash value
algo = arguments.algorithm
if algo.lower() in ("md5", "sha1", "sha224", "sha384", "sha512"):
for hashValue in hashFile(algo, arguments.filepaths):
print hashValue
else:
print "Algorithm {0} is not available in this script".format(algorithm)
if __name__ == "__main__":
main()
I haven't tried it in Python 3, but I get the same error in Python 2.7.5 for binary files (the only difference is that mine is with the ascii codec). Instead of encoding the data chunks, open the file directly in binary mode:
with open(path, 'rb') as f:
while True:
dataChunk = f.read(blockSize)
if not dataChunk:
break
algorithmType.update(dataChunk)
yield algorithmType.hexdigest()
Apart from that, I'd use the method hashlib.new instead of getattr, and hashlib.algorithms_available to check if the argument is valid.

Getting all reviews from a specific Windows Phone Marketplace Game

Does AppHub let us see reviews of our apps from all marketplaces at once? As I didn't find any, o took some time writing some code to print them all in a file, so i won't waste my time looking for them in every single language.
I'd appreciate any better solution. In the worst case, I'm glad to share the code with anyone who finds it usefull.
It uses BeautifulSoup.
The only parametter is the id of the app, like this:
wp7reviews.py 62289160-6970-4674-85a0-aef3dbe3f93d
Here is the code
import sys
import getopt
from urllib2 import URLError
from urllib2 import HTTPError
import urllib2
from BeautifulSoup import BeautifulStoneSoup
opts, extraparams = getopt.getopt(sys.argv[1:], '')
# starts at the second element of argv since the first one is the script name
# extraparms are extra arguments passed after all option/keywords are assigned
# opts is a list containing the pair "option"/"value"
#print 'Opts:',opts
#print 'Extra parameters:',extraparams
try:
appid = extraparams[0]
except:
#Awsome Linkit appid as default appid
appid="62289160-6970-4674-85a0-aef3dbe3f93d"
allreviewsFILE = open("allreviews.txt", "w")
def output(text):
allreviewsFILE.write(text)
#print text,
def outputln(text):
allreviewsFILE.write(text+'\n')
#print text
def geturl(lang):
return "http://catalog.zune.net/v3.2/"+lang+"/apps/"+appid
try:
request = urllib2.Request(geturl("en-us"))
fd = urllib2.urlopen(request)
content = fd.read()
fd.close()
soup = BeautifulStoneSoup(content)
try:
outputln("App title: "+soup.findAll("a:title")[0].string)
outputln("");
except:
print "Failed to get App Title"
langs = ["en-us", "en-gb", "de-de",
"fr-fr", "es-es", "it-it",
"en-au", "de-at", "fr-be",
"fr-ca", "en-ca", "en-hk",
"en-in", "en-ie", "es-mx",
"en-nz", "en-sg", "de-ch",
"fr-ch", "zh-hk", "zh-cn",
"en-hk"]
outputln("Here we got reviews from each marketplace")
for lang in langs:
request = urllib2.Request(geturl(lang)+"/reviews")
fd = urllib2.urlopen(request)
print "Fetching "+lang+"...",
content = fd.read()
fd.close()
print "OK"
soup = BeautifulStoneSoup(content)
#print soup.prettify()
contents = soup.findAll("a:content")
ratings = soup.findAll("userrating")
l = len(contents)
if l > 0:
outputln("----------- "+lang+" ---------------------------------------------------")
outputln("")
for i in range(0, l):
output(ratings[i].string+"/10 - ")
if len(contents[i].contents) > 0:
try:
outputln(contents[i].contents[0])
except:
outputln("*** Unknown chars ***")
else:
outputln("Rating only")
outputln("")
except HTTPError, e:
print("Error during request!\n")
print("Cod.: ", e.code)
except URLError, e:
print("Invalid URL!\n")
print("Message: ", e.reason)
There already is a site that gives you this information. Take a look at http://wp7reviews.tomverhoeff.com/
There is also a free WP7 app called AppTracker which allows you to track reviews from different regions, as well as translate them into your native language

Resources