I'm writing Scala code. What is the correct path schema for writing a URI when using Rapture to operate with files on Windows? I have added the following dependencies:
libraryDependencies += "com.propensive" %% "rapture" % "2.0.0-M3" exclude("com.propensive","rapture-json-lift_2.11")
Here is part of my code:
import rapture.uri._
import rapture.io._
val file = uri"file:///C:/opt/eric/spark-demo"
file.delete()
but I got the message:
Error:(17, 16) value file is not a member of object rapture.uri.UriContext
val file = uri"file:///C:/opt/eric/spark-demo"
or I tried this one:
val file = uri"file://opt/eric/spark-demo"
The same error:
Error:(17, 16) value file is not a member of object rapture.uri.UriContext
val file = uri"file://opt/eric/spark-demo"
And my local path is:
C:\opt\eric\spark-demo
How can I avoid the error?
You're missing an import:
import rapture.io._
import rapture.uri._
import rapture.fs._
val file = uri"file:///C:/opt/eric/spark-demo"
file.delete()
rapture.fs is the package defining an EnrichedFileUriContext implicit class, which is what the uri macro trys to build when being provided with a file URI scheme.
Related
After testing the logger library locally, I uploaded it to pypi.
Afterwards, when I proceeded with pip install, there was an error saying that the module inside the library could not be found.
So, as a temporary measure, I added a syntax to register all .py in init.py in the library package folder, and I want to improve this. This is because you have to install all dependencies for features that users may not be using
What improvements can I take in this situation?
If possible, I would like to know how to lazy use only the modules used by the user instead of registering all .py in init.py .
Or is there something structurally I'm overlooking?
Here is the project structure I used
project_name
- pacakge_name
- __init__.py. <- all loggers were registered
- file_logger.py
- console_logger.py
- ...
- fluent_logger.py <- used external library
- scribe_logger.py <- used external library
init.py
"""
Description for Package
"""
from .composite_logger import CompositeLogger
from .console_logger import ConsoleLogger
from .file_logger import FileLogger
from .fluent_logger import FluentLogger
from .jandi_logger import JandiLogger
from .line_logger import LineLogger
from .logger_impl import LoggerImpl
from .logger_interface import LoggerInterface
from .logger import Logger
from .memory_logger import MemoryLogger
from .null_logger import NullLogger
from .scribe_logger import ScribeLogger
from .telegram_logger import TelegramLogger
from .retry import Retry
__all__ = [
'CompositeLogger',
'ConsoleLogger',
'FileLogger',
'FluentLogger',
'JandiLogger',
'LineLogger',
'LoggerImpl',
'LoggerInterface',
'Logger',
'MemoryLogger',
'NullLogger',
'ScribeLogger',
'TelegramLogger',
'Retry',
]
setup.py
import setuptools
from distutils.core import setup
with open("README.md", "r", encoding="utf-8") as f:
long_descriprion = f.read()
setuptools.setup(
name = 'project_name',
version = '0.0.1',
description = 'python logger libary',
long_description = long_descriprion,
long_description_content_type = "text/markdown",
author = 'abc',
author_email = 'abc#gmail.com',
url = "https://github.com/##/##",
packages = ["pacakge_name"],
install_requires=[ <- contains too many external libraries
'requests>=2.0.0',
'thrift>=0.16.0',
'facebook-scribe>=2.0.post1',
'fluent-logger>=0.10.0'
],
keywords = ['logger'],
python_requires = '>=3.7',
classifiers = [
'Programming Language :: Python :: 3.7',
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent"
],
)
I work with a Mac. I have been trying to make a multiple sequence alignment in Python using Muscle. This is the code I have been running:
from Bio.Align.Applications import MuscleCommandline
cline = MuscleCommandline(input="testunaligned.fasta", out="testunaligned.aln", clwstrict=True)
print(cline)
from Bio import AlignIO
align = AlignIO.read(open("testunaligned.aln"), "clustal")
print(align)
I keep getting the following error:
FileNotFoundError: [Errno 2] No such file or directory: 'testunaligned.aln'
Does anyone know how I could fix this? I am very new to Python and computer science in general, and I am totally at a loss. Thanks!
cline in your code is an instance of MuscleCommandline object that you initialized with all the parameters. After the initialization, this instance can run muscle, but it will only do that if you call it. That means you have to invoke cline()
When you simply print the cline object, it will return a string that corresponds to the command you can manually run on the command line to get the same result as when you invoke cline().
And here the working code:
from Bio.Align.Applications import MuscleCommandline
cline = MuscleCommandline(
input="testunaligned.fasta",
out="testunaligned.aln",
clwstrict=True
)
print(cline)
cline() # this is where mucle runs
from Bio import AlignIO
align = AlignIO.read(open("testunaligned.aln"), "clustal")
print(align)
This a section in a schema file
imports:
- path: configs/folder1/resources/gcpresource/test/*
I am trying to import all the files in a folder using the template's schema file.
I know that this does not work.
My question is,
what is a better way to import all the files using * or something that's suitable so that deployment manager can import all of the files in a folder without having to explicitly specify them ?
You could do so in python if they are all yaml config files
# my-template.yaml
imports:
- path: omni-importer.py
And in the middleware-like python template do something like:
# omni-importer.py
import yaml
from os import listdir
from os.path import isfile, join
def generate_config(ctx):
mypath = ctx.properties['path']
files = [f for f in listdir(mypath) if isfile(join(mypath, f)) and f.endswith('yaml')]
yamls = map(lambda f: yaml.load(open('test.txt')['resources']), files)
return {
'resources': yamls,
}
This is not working code, but rather a proof of concept
Im trying to use protobuffers and scalaPB, i've added:
import com.trueaccord.scalapb.compiler.Version.scalapbVersion
libraryDependencies += "com.trueaccord.scalapb" %% "scalapb-runtime" %
scalapbVersion % "protobuf"
PB.targets in Compile := Seq(
scalapb.gen() -> (sourceManaged in Compile).value
)
and below lines to plugins.sbt
addSbtPlugin("com.thesamet" % "sbt-protoc" % "0.99.3")
libraryDependencies += "com.trueaccord.scalapb" %% "compilerplugin" % "0.5.47"
and .proto file under src/main/protobuf:
syntax = "proto3";
import "scalapb/scalapb.proto";
import "google/protobuf/wrappers.proto";
package actors;
message ExamplePROTO {
double value = 1;
}
but using sbt compile nothing happens - class is not generated in target. File is ignored, because even with some typo in protobuf file project still compiles. Ive also tried to look at debug logs using
logLevel in Global := Level.Debug
but the only files .proto which are mentioned there are some internal ones like: /target/protobuf_external/google/protobuf/source_context.proto
How can I proceed with it? Can I get some more information what scalaPB is looking at?
I'm trying to run a code from github that uses Python to classify images but I'm getting an error.
here is the code:
import argparse as ap
import cv2
import imutils
import numpy as np
import os
from sklearn.svm import LinearSVC
from sklearn.externals import joblib
from scipy.cluster.vq import *
# Get the path of the testing set
parser = ap.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument("-t", "--testingSet", help="Path to testing Set")
group.add_argument("-i", "--image", help="Path to image")
parser.add_argument('-v',"--visualize", action='store_true')
args = vars(parser.parse_args())
# Get the path of the testing image(s) and store them in a list
image_paths = []
if args["testingSet"]:
test_path = args["testingSet"]
try:
testing_names = os.listdir(test_path)
except OSError:
print "No such directory {}\nCheck if the file exists".format(test_path)
exit()
for testing_name in testing_names:
dir = os.path.join(test_path, testing_name)
class_path = imutils.imlist(dir)
image_paths+=class_path
else:
image_paths = [args["image"]]
and this is the error message I'm getting
usage: getClass.py [-h]
(- C:/Users/Lenovo/Downloads/iris/bag-of-words-master/dataset/test TESTINGSET | - C:/Users/Lenovo/Downloads/iris/bag-of-words-master/dataset/test/test_1.jpg IMAGE)
[- C:/Users/Lenovo/Downloads/iris/bag-of-words-master/dataset]
getClass.py: error: one of the arguments - C:/Users/Lenovo/Downloads/iris/bag-of-words-master/dataset/test/--testingSet - C:/Users/Lenovo/Downloads/iris/bag-of-words-master/dataset/test/test_1.jpg/--image is required
can you please help me with this? where and how should I write the file path?
This is an error your own program is issuing. The message is not about the file path but about the number of arguments. This line
group = parser.add_mutually_exclusive_group(required=True)
says that only one of your command-line arguments (-t, -i) is permitted. But it appears from the error message that you are supplying both --testingSet and --image on your command line.
Since you only have 3 arguments, I have to wonder if you really need argument groups at all.
To get your command line to work, drop the mutually-exclusive group and add the arguments to the parser directly.
parser.add_argument("-t", "--testingSet", help="Path to testing Set")
parser.add_argument("-i", "--image", help="Path to image")
parser.add_argument('-v',"--visualize", action='store_true')