This a section in a schema file
imports:
- path: configs/folder1/resources/gcpresource/test/*
I am trying to import all the files in a folder using the template's schema file.
I know that this does not work.
My question is,
what is a better way to import all the files using * or something that's suitable so that deployment manager can import all of the files in a folder without having to explicitly specify them ?
You could do so in python if they are all yaml config files
# my-template.yaml
imports:
- path: omni-importer.py
And in the middleware-like python template do something like:
# omni-importer.py
import yaml
from os import listdir
from os.path import isfile, join
def generate_config(ctx):
mypath = ctx.properties['path']
files = [f for f in listdir(mypath) if isfile(join(mypath, f)) and f.endswith('yaml')]
yamls = map(lambda f: yaml.load(open('test.txt')['resources']), files)
return {
'resources': yamls,
}
This is not working code, but rather a proof of concept
Related
After testing the logger library locally, I uploaded it to pypi.
Afterwards, when I proceeded with pip install, there was an error saying that the module inside the library could not be found.
So, as a temporary measure, I added a syntax to register all .py in init.py in the library package folder, and I want to improve this. This is because you have to install all dependencies for features that users may not be using
What improvements can I take in this situation?
If possible, I would like to know how to lazy use only the modules used by the user instead of registering all .py in init.py .
Or is there something structurally I'm overlooking?
Here is the project structure I used
project_name
- pacakge_name
- __init__.py. <- all loggers were registered
- file_logger.py
- console_logger.py
- ...
- fluent_logger.py <- used external library
- scribe_logger.py <- used external library
init.py
"""
Description for Package
"""
from .composite_logger import CompositeLogger
from .console_logger import ConsoleLogger
from .file_logger import FileLogger
from .fluent_logger import FluentLogger
from .jandi_logger import JandiLogger
from .line_logger import LineLogger
from .logger_impl import LoggerImpl
from .logger_interface import LoggerInterface
from .logger import Logger
from .memory_logger import MemoryLogger
from .null_logger import NullLogger
from .scribe_logger import ScribeLogger
from .telegram_logger import TelegramLogger
from .retry import Retry
__all__ = [
'CompositeLogger',
'ConsoleLogger',
'FileLogger',
'FluentLogger',
'JandiLogger',
'LineLogger',
'LoggerImpl',
'LoggerInterface',
'Logger',
'MemoryLogger',
'NullLogger',
'ScribeLogger',
'TelegramLogger',
'Retry',
]
setup.py
import setuptools
from distutils.core import setup
with open("README.md", "r", encoding="utf-8") as f:
long_descriprion = f.read()
setuptools.setup(
name = 'project_name',
version = '0.0.1',
description = 'python logger libary',
long_description = long_descriprion,
long_description_content_type = "text/markdown",
author = 'abc',
author_email = 'abc#gmail.com',
url = "https://github.com/##/##",
packages = ["pacakge_name"],
install_requires=[ <- contains too many external libraries
'requests>=2.0.0',
'thrift>=0.16.0',
'facebook-scribe>=2.0.post1',
'fluent-logger>=0.10.0'
],
keywords = ['logger'],
python_requires = '>=3.7',
classifiers = [
'Programming Language :: Python :: 3.7',
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent"
],
)
I'm currently using ansible 2.3.3 and trying to upgrade it to a newever version but since 2.4+ the ansible.inventory.Inventory class has been removed.
Does anybody know what is the equivalent recommended replacement of ansible.inventory.Inventory in Ansible 2.4+?
from ansible.inventory import Inventory
from ansible.parsing.dataloader import DataLoader
from ansible.vars import VariableManager
inv = Inventory(
loader = DataLoader(),
variable_manager = VariableManager(),
host_list = my_path,
)
There isn't a one to one replacement in the newer code but the following code can be used but note that the inventory now has a different data structure
from ansible.inventory.manager import InventoryManager
from ansible.parsing.dataloader import DataLoader
m = InventoryManager(loader=DataLoader(), sources='/path/to/inventory')
m.hosts
m.groups
I used cachix for nix if it has any bearing, and reinstalled stack, too. The repo in question is at: https://github.com/NorfairKing/tickler
[jk#jk tickler]$ stack build
Cloning 85ee8a577fb576e2dd7643bf248ff8fbbe9598ec from git#github.com:NorfairKing/pretty-relative-time.git
Enter passphrase for key '/home/jk/.ssh/id_rsa':
Unsupported tarball from /tmp/with-repo-archive30826/foo.tar: File located at "result" is a symbolic link to absolute path /nix/store/j0l1f389pmxdazxr0h0ax8v6c8ln578h-pretty-relative-time-0.0.0.0
Running nix build:
[jk#jk tickler]$ nix build
error: Please be informed that this pseudo-package is not the only part of
Nixpkgs that fails to evaluate. You should not evaluate entire Nixpkgs
without some special measures to handle failing packages, like those taken
by Hydra.
let
pkgsv = import (import ./nix/nixpkgs.nix);
pkgs = pkgsv {};
intray-version = import ./nix/intray-version.nix;
intray-repo = pkgs.fetchFromGitHub intray-version;
intray-overlay = import (intray-repo + "/nix/overlay.nix");
validity-version = import (intray-repo + "/nix/validity-version.nix");
validity-overlay = import (pkgs.fetchFromGitHub validity-version + "/nix/overlay.nix");
in pkgsv {
overlays = [ validity-overlay intray-overlay (import ./nix/overlay.nix) ];
config.allowUnfree = true;
}
TL,DR: From a Sphinx extension, how do I tell sphinx-build to treat an additional file as a dependency? In my immediate use case, this is the extension's source code, but the question could equally apply to some auxiliary file used by the extension.
I'm generating documentation with Sphinx using a custom extension. I'm using sphinx-build to build the documentation. For example, I use this command to generate the HTML (this is the command in the makefile generated by sphinx-quickstart):
sphinx-build -b html -d _build/doctrees . _build/html
Since my custom extension is maintained together with the source of the documentation, I want sphinx-build to treat it as a dependency of the generated HTML (and LaTeX, etc.). So whenever I change my extension's source code, I want sphinx-build to regenerate the output.
How do I tell sphinx-build to treat an additional file as a dependency? That is not mentioned in the toctree, since it isn't part of the source. Logically, this should be something I do from my extension's setup function.
Sample extension (my_extension.py):
from docutils import nodes
from docutils.parsers.rst import Directive
class Foo(Directive):
def run(self):
node = nodes.paragraph(text='Hello world\n')
return [node]
def setup(app):
app.add_directive('foo', Foo)
Sample source (index.rst):
.. toctree::
:maxdepth: 2
.. foo::
Sample conf.py (basically the output of sphinx-quickstart plus my extension):
import sys
import os
sys.path.insert(0, os.path.abspath('.'))
extensions = ['my_extension']
templates_path = ['_templates']
source_suffix = '.rst'
master_doc = 'index'
project = 'Hello directive'
copyright = '2019, Gilles'
author = 'Gilles'
version = '1'
release = '1'
language = None
exclude_patterns = ['_build']
pygments_style = 'sphinx'
todo_include_todos = False
html_theme = 'alabaster'
html_static_path = ['_static']
htmlhelp_basename = 'Hellodirectivedoc'
latex_elements = {
}
latex_documents = [
(master_doc, 'Hellodirective.tex', 'Hello directive Documentation',
'Gilles', 'manual'),
]
man_pages = [
(master_doc, 'hellodirective', 'Hello directive Documentation',
[author], 1)
]
texinfo_documents = [
(master_doc, 'Hellodirective', 'Hello directive Documentation',
author, 'Hellodirective', 'One line description of project.',
'Miscellaneous'),
]
Validation of a solution:
Run make html (or sphinx-build as above).
Modify my_extension.py to replace Hello world by Hello again.
Run make html again.
The generated HTML (_build/html/index.html) must now contain Hello again instead of Hello world.
It looks like the note_dependency method in the build environment API should do what I want. But when should I call it? I tried various events but none seemed to hit the environment object in the right state. What did work was to call it from a directive.
import os
from docutils import nodes
from docutils.parsers.rst import Directive
import sphinx.application
class Foo(Directive):
def run(self):
self.state.document.settings.env.note_dependency(__file__)
node = nodes.paragraph(text='Hello done\n')
return [node]
def setup(app):
app.add_directive('foo', Foo)
If a document contains at least one foo directive, it'll get marked as stale when the extension that introduces this directive changes. This makes sense, although it could get tedious if an extension adds many directives or makes different changes. I don't know if there's a better way.
Inspired by Luc Van Oostenryck's autodoc-C.
As far as I know app.env.note_dependency can be called within the doctree-read to add any file as a dependency to the document currently being read.
So in your use case, I assume this would work:
from typing import Any, Dict
from sphinx.application import Sphinx
import docutils.nodes as nodes
def doctree-read(app: Sphinx, doctree: nodes.document):
app.env.note_dependency(file)
def setup(app: Sphinx):
app.connect("doctree-read", doctree-read)
I'm writing Scala code. What is the correct path schema for writing a URI when using Rapture to operate with files on Windows? I have added the following dependencies:
libraryDependencies += "com.propensive" %% "rapture" % "2.0.0-M3" exclude("com.propensive","rapture-json-lift_2.11")
Here is part of my code:
import rapture.uri._
import rapture.io._
val file = uri"file:///C:/opt/eric/spark-demo"
file.delete()
but I got the message:
Error:(17, 16) value file is not a member of object rapture.uri.UriContext
val file = uri"file:///C:/opt/eric/spark-demo"
or I tried this one:
val file = uri"file://opt/eric/spark-demo"
The same error:
Error:(17, 16) value file is not a member of object rapture.uri.UriContext
val file = uri"file://opt/eric/spark-demo"
And my local path is:
C:\opt\eric\spark-demo
How can I avoid the error?
You're missing an import:
import rapture.io._
import rapture.uri._
import rapture.fs._
val file = uri"file:///C:/opt/eric/spark-demo"
file.delete()
rapture.fs is the package defining an EnrichedFileUriContext implicit class, which is what the uri macro trys to build when being provided with a file URI scheme.