When setting up a github actions pipeline, I can't get it to find packages that are within my repository, and the test fails because it's missing packages.
What happens is that it clones the repo someplace but doesn't include the cloned repo's directories to look for packages. That fails because I am importing packages from within that repo in my code.
I believe my directory structure is sound because I have no trouble testing and building locally:
. │
├── extractors │
│ ├── fip.go │
│ └── fip_test.go │
├── fixtures │
│ └── fip │
│ ├── bad_req.json │
│ └── history_response.json │
├── .github │
│ └── workflows │
│ └── go_test.yml │
├── main.go │
├── Makefile │
├── playlist │
│ └── playlist.go │
├── README.md │
└── utils │
├── logger │
│ └── logger.go │
└── mocks │
└── server.go │
│
View the run here
How do I make Github actions look for the package within the cloned dir as well?
Make sure to run go mod init MODULE_NAME (if the project is outside GOROOT or GOPATH) or just simply go mod init (if the project is inside GOROOT or GOPATH). The command should be run on the root folder of your project. This would create a go.mod file that would enable go resolve your packages.
Related
Are there any best-practices how to organize your project folders so that the CI/CD pipline remains simple?
Here, the following structure is used, which seems to be quite complex:
project
│ README.md
│ azure-pipelines.yml
│ config.json
│ .gitignore
└─── package1
│ │ __init__.py
│ │ setup.py
│ │ README.md
│ │ file.py
│ └── submodule
│ │ │ file.py
│ │ │ file_test.py
│ └── requirements
│ │ │ common.txt
│ │ │ dev.txt
│ └─ notebooks
│ │ notebook1.txt
│ │ notebook2.txt
└─── package2
| │ ...
└─── ci_cd_scripts
│ requirements.py
│ script1.py
│ script2.py
│ ...
Here, the following structure is suggested:
.
├── .dbx
│ └── project.json
├── .github
│ └── workflows
│ ├── onpush.yml
│ └── onrelease.yml
├── .gitignore
├── README.md
├── conf
│ ├── deployment.json
│ └── test
│ └── sample.json
├── pytest.ini
├── sample_project
│ ├── __init__.py
│ ├── common.py
│ └── jobs
│ ├── __init__.py
│ └── sample
│ ├── __init__.py
│ └── entrypoint.py
├── setup.py
├── tests
│ ├── integration
│ │ └── sample_test.py
│ └── unit
│ └── sample_test.py
└── unit-requirements.txt
In concrete, I want to know:
Should I use one repo for all repositories and notebooks (such as suggested in the first approach) or should I create one repo per library (which makes the CI/CD more effortfull as there might be dependencies between the packages)
With both suggested folder structures it is unclear for me where to place my notebooks that are not related to any specific package (e.g. notebooks that contain my business logic and use the package)?
Is there a well-established folder structure?
The Databricks had a repository with project templates to be used with Databricks (link) but now it has been archived and the template creation is part of dbx tool - maybe these two links will be useful for you:
dbx init command - https://dbx.readthedocs.io/en/latest/reference/cli/?h=init#dbx-init
DevOps for Workflows Guide - https://dbx.readthedocs.io/en/latest/concepts/devops/#devops-for-workflows
Let's say I have a project called my-project/ that lives in it's own directory and has the following file structure.
my-project/
.
├── src
│ ├── index.html
│ ├── main.js
│ ├── normalize.js
│ ├── routes
│ │ ├── index.js
│ │ └── Home
│ │ ├── index.js
│ │ └── assets
│ ├── static
│ ├── store
│ │ ├── createStore.js
│ │ └── reducers.js
│ └── styles
└── project.config.js
Now let's say I have a new project called my-new-project that also lives in it's own directory and has the same file structure as my-project but it contains an additional file called my-files-to-copy.txt
my-new-project/
.
├── src
│ ├── index.html
│ ├── main.js
│ ├── normalize.js
│ ├── routes
│ │ ├── index.js
│ │ └── Home
│ │ ├── index.js
│ │ └── assets
│ ├── static
│ ├── store
│ │ ├── createStore.js
│ │ └── reducers.js
│ └── styles
├── project.config.js
└── my-files-to-copy.txt # new file added to tree
my-new-project/ has the same file structure but different file contents than my-project/
Now let's say my-files-to-copy.txt contains a list of files I want to copy from my-project/ and write to the same path in my-new-project/ to overwrite the existing files in my-new-project/ at those locations.
my-files-to-copy.txt
src/main.js
src/routes/index.js
src/store/reducers.js
project.config.js
How can I accomplish this with a terminal/bash/shell command or script?
edit:
I think I might be able to do:
cp my-project/src/main.js my-new-project/src/main.js
cp my-project/src/routes/index.js my-new-project/src/routes/index.js
cp my-project/src/store/reducers.js my-new-project/src/store/reducers.js
cp my-project/project.config.js my-new-project/project.config.js
But as the number of files scales, this method will become less efficient. I was looking for a more efficient solution that would allow me to leverage the file that contains the list of files (or at least a script) without having to write a separate command for each one.
Assuming my-project and my-new-project are on the same directory:
xargs -i -a my-new-project/my-files-to-copy.txt cp my-project/{} my-new-project/{}
In our project we create docker images, push to registry using fabric8 maven plugin. We have a requirement in which we need to find out the details of each image (name, tag , registry pushed to etc) that gets created in the project build.
Is there any out of the box solution in fabric8 for that ? or do we have any java API's of fabric8 available which we can use and integrate in our existing maven plugin to get such info?
As far as I know, there is no way to get such info from the plugin itself. But when building Docker images with fabric8's docker-maven-plugin, a hierarchy like the following is created on the target/ folder by default :
target
├── docker
│ ├── build.timestamp
│ └── my
│ └── registry
│ ├── image1
│ │ ├── tag1
│ │ │ ├── build
│ │ │ │ └── Dockerfile
│ │ │ ├── tmp
│ │ │ │ └── docker-build.tar
│ │ │ └── work
│ │ └── tag2
│ │ ├── build
│ │ │ └── Dockerfile
│ │ ├── tmp
│ │ │ └── docker-build.tar
│ │ └── work
│ ├── image2
│ │ ├── tag1
│ │ │ ├── ...
... ... ... ...
In this example, 3 images were built :
my/registry/image1:tag1
my/registry/image1:tag2
my/registry/image2:tag1
So browsing this hierarchy can inform you about the images names, tags and registries. I agree that it is not the cleanest solution.
Alternatively, you can use spotify's dockerfile-maven-plugin (https://github.com/spotify/dockerfile-maven), because it creates a docker-info JAR containing what you're looking for :
META-INF/
META-INF/MANIFEST.MF
META-INF/docker/
META-INF/docker/my/registry/
META-INF/docker/my/registry/image1/
META-INF/docker/my/registry/image1/image-name
META-INF/docker/my/registry/image1/repository
META-INF/docker/my/registry/image1/tag
META-INF/docker/my/registry/image1/image-id
META-INF/maven/
META-INF/maven/my/registry/
META-INF/maven/my/registry/image1/
META-INF/maven/my/registry/image1/pom.xml
META-INF/maven/my/registry/image1/pom.properties
This docker-info JAR could then be put into a repository manager to help retrieving information about built images, and moreover to version your different builds.
I am trying to launch JBoss application server - my goal is to launch it and deploy some very simple project (I'm trying to do this with "helloworld" from the original quickstarts). The problem is that I have no experience with JBoss or Maven, so I'm having terrible time for a few days and it still isn't working. I presume, that the mistake is in Maven configuration, but I don't know, what I'm supposed to rewrite / replace to repair it.
This is the exact error:
[INFO] Error building POM (may not be this project's POM).
Project ID: org.jboss.component.management:jboss-dependency-management-all
Reason: POM 'org.jboss.component.management:jboss-dependency-management-all' not found in repository: Unable to download the artifact from any repository
org.jboss.component.management:jboss-dependency-management-all:pom:6.0.1-redhat-1
from the specified remote repositories:
central (http://repo1.maven.org/maven2)
for project org.jboss.component.management:jboss-dependency-management-all
I was trying to follow instructions given at http://www.jboss.org/quickstarts/eap/#build-and-deploy-th%20e-quickstarts , so my only configuration of Maven was, that I copied settings.xml from quickstart directory to .m2 directory.Finally I tried to build and deploy quickstart by command "mvn clean install jboss-as:deploy", but it caused the error :-(
How to repair this mistake?
P.s.: I use Ubuntu 14.04.
This is how the structure of my folder with JBoss looks like:
.
├── InstallationLog.txt
├── InstallSummary.html
├── jboss-eap-6.2
│ ├── appclient
│ ├── bin
│ ├── bundles
│ ├── docs
│ ├── domain
│ ├── icons
│ ├── JBossEULA.txt
│ ├── jboss-modules.jar
│ ├── LICENSE.txt
│ ├── modules
│ ├── standalone
│ ├── version.txt
│ └── welcome-content
├── jboss-eap-6.2.0.GA-quickstarts
│ ├── bean-validation
│ ├── bmt
│ ├── cdi-alternative
│ ├── cdi-decorator
│ ├── cdi-injection
│ ├── cdi-interceptors
│ ├── cdi-portable-extension
│ ├── cdi-stereotype
│ ├── cdi-veto
│ ├── cluster-ha-singleton
│ ├── cmt
│ ├── configure-postgresql.cli
│ ├── CONTRIBUTING.html
│ ├── CONTRIBUTING.md
│ ├── contributor-settings.xml
│ ├── dist
│ ├── ejb-asynchronous
│ ├── ejb-in-ear
│ ├── ejb-in-war
│ ├── ejb-multi-server
│ ├── ejb-remote
│ ├── ejb-security
│ ├── ejb-security-interceptors
│ ├── ejb-throws-exception
│ ├── ejb-timer
│ ├── forge-from-scratch
│ ├── greeter
│ ├── guide
│ ├── helloworld
│ ├── helloworld-jms
│ ├── helloworld-mbean
│ ├── helloworld-mdb
│ ├── helloworld-osgi
│ ├── helloworld-rs
│ ├── helloworld-singleton
│ ├── helloworld-ws
│ ├── hibernate3
│ ├── hibernate4
│ ├── hornetq-clustering
│ ├── h2-console
│ ├── inter-app
│ ├── jax-rs-client
│ ├── jta-crash-rec
│ ├── jts
│ ├── jts-distributed-crash-rec
│ ├── kitchensink
│ ├── kitchensink-ear
│ ├── kitchensink-jsp
│ ├── kitchensink-ml
│ ├── kitchensink-ml-ear
│ ├── LICENSE.txt
│ ├── logging
│ ├── logging-tools
│ ├── log4j
│ ├── mail
│ ├── numberguess
│ ├── payment-cdi-event
│ ├── picketlink-sts
│ ├── pom.xml
│ ├── README.html
│ ├── README.md
│ ├── RELEASE_PROCEDURE.html
│ ├── RELEASE_PROCEDURE.md
│ ├── remove-postgresql.cli
│ ├── servlet-async
│ ├── servlet-filterlistener
│ ├── servlet-security
│ ├── settings.xml
│ ├── shopping-cart
│ ├── tasks
│ ├── tasks-jsf
│ ├── tasks-rs
│ ├── temperature-converter
│ ├── template
│ ├── wicket-ear
│ ├── wicket-war
│ ├── wsat-simple
│ ├── wsba-coordinator-completion-simple
│ ├── wsba-participant-completion-simple
│ ├── xml-dom4j
│ └── xml-jaxp
└── Uninstaller
└── uninstaller.jar
settings.xml: http://hostcode.sourceforge.net/view/1926
pom.xml: http://hostcode.sourceforge.net/view/1927
Eventually I've solved this - thorough delete of all Maven files ( or files which held some deprecated info about Maven ) and installation of its newest version was the key :-)
This solution is written for Ubuntu 14.04 LTS (but it should work in other versions and even other Debian-based systems too).
STEP-BY-STEP SOLUTION:
1) Completely uninstall Maven
sudo apt-get purge maven
2) Uninstall libaether package
sudo apt-get purge libaether-java
3) Uninstall libplexus packages
sudo apt-get purge libplexus-*
4) Finally, install actual Maven version from the repository
sudo apt-get install maven
I read in wiki that NoScript is open source http://en.wikipedia.org/wiki/NoScript, but on official site http://noscript.net/, I can't find any sources. So my question is: where to find sources? Or, is there something I did not understand, and the source code is not available?
The Firefox XPI format does not prevent you from simply extracting the contents of the plugin to examine the source code.
While I cannot find a canonical public repository, it looks like someone has systematically downloaded and extracted all the available XPIs and created a GitHub repository out of them.
https://github.com/avian2/noscript
If you'd like to do it yourself, XPI files are just standard ZIP files, so if you want to extract one yourself you can simply point an extraction program at it.
Here's an example of doing that from the command line:
mkdir noscript_source
cd noscript_source
curl -LO https://addons.mozilla.org/firefox/downloads/file/219550/noscript_security_suite-2.6.6.8-fx+fn+sm.xpi
unzip noscript_security_suite-2.6.6.8-fx+fn+sm.xpi
That yields a directory structure that looks like this:
.
├── GPL.txt
├── META-INF
│ ├── manifest.mf
│ ├── zigbert.rsa
│ └── zigbert.sf
├── NoScript_License.txt
├── chrome
│ └── noscript.jar
├── chrome.manifest
├── components
│ └── noscriptService.js
├── defaults
│ └── preferences
│ └── noscript.js
├── install.rdf
├── mozilla.cfg
└── noscript_security_suite-2.6.6.8-fx+fn+sm.xpi
Then the main code is located inside chrome/noscript.jar. You can extract that to get at the JavaScript that makes up the bulk of the plugin:
cd chrome/
unzip noscript.jar
Which will yield the main source tree:
.
├── content
│ └── noscript
│ ├── ABE.g
│ ├── ABE.js
│ ├── ABELexer.js
│ ├── ABEParser.js
│ ├── ASPIdiocy.js
│ ├── ChannelReplacement.js
│ ├── ClearClickHandler.js
│ ├── ClearClickHandlerLegacy.js
│ ├── Cookie.js
│ ├── DNS.js
│ ├── DOM.js
│ ├── ExternalFilters.js
│ ├── FlashIdiocy.js
│ ├── HTTPS.js
│ ├── Lang.js
│ ├── NoScript_License.txt
│ ├── PlacesPrefs.js
│ ├── Plugins.js
│ ├── Policy.js
│ ├── Profiler.js
│ ├── Removal.js
│ ├── RequestWatchdog.js
│ ├── STS.js
│ ├── ScriptSurrogate.js
│ ├── Strings.js
│ ├── URIValidator.js
│ ├── about.xul
│ ├── antlr.js
│ ├── clearClick.js
│ ├── clearClick.xul
│ ├── frameOptErr.xhtml
│ ├── iaUI.js
│ ├── noscript.js
│ ├── noscript.xbl
│ ├── noscriptBM.js
│ ├── noscriptBMOverlay.xul
│ ├── noscriptOptions.js
│ ├── noscriptOptions.xul
│ ├── noscriptOverlay.js
│ ├── noscriptOverlay.xul
│ ├── options-mobile.xul
│ └── overlay-mobile.xul
├── locale
└── skin
The extension contains the source code - you just need to unzip it. See Giorgio's response here.
The whole source code is publicly available in every each XPI.
You've got it on your hard disk right now, if you're a NoScript user, otheriwise you can download it here.
You can examine and/or modify it by unzipping the XPI and the JAR inside, and "building" it back by rezipping both.
It's been like that for ever, since the very first version.