I have a gitlab installation for my organization. To standardize, the frameworks and development process we'd like to make the use of our own packagist installation on our server.
The thing that is confusing me, is when I have to submit a package which is on a repo of gitlab, it requires authentication. I know while including a package in composer.json I can put repo, username and password so that it can read package from there. or while I am creating a project:
composer create-project vendor/name path --repository-url=http://user:password#my.gitlab.com/myrepo/package.git
If that is true, do we really need packagist? if yes then what would be the best way to setup this.
I'd appreciate if you guys also tell me how the package submission on packagist(our own hosted) will work.
Related
I am using artifactory and have no issues with the npm solution. However, I have set up composer using the "Set Me Up" instructions, i.e. installed the ~/.composer/config.json & ~/.composer/auth.json files. I now get the following error when I try to perform a composer install:
$ dcomp install
Loading composer repositories with package information
[Composer\Downloader\TransportException]
The "https://myartifactory.jfrog.io/artifactory/api/composer/php-local/packages.json" file could not be downloaded (HTTP/1.1 404 Not Found)
install [--prefer-source] [--prefer-dist] ...
The user I have configured in auth.json is an admin user, and I've pushed a single package into the Artifactory. Additionally, I noticed the url contains plural packages.json as opposed to singular package.json, so I am assuming this is an issue with the artifactory configuration itself and not the package I've uploaded.
For Artifactory to index composer packages successfully you also need to make sure that each package has version specified.
There are 3 ways to achieve that:
Include the version attribute in the package composer.json file
Set a composer.version property on the package itself.
Use the version field when deploying it via the UI.
You can easily test it inside the UI by clicking on the archive you just uploaded, then go to the properties tab and add a property for example:
composer.version with value for example 1.0.0.
The issue in my case was as a couple commenters observed: the index wasn't being generated.
For me, the fix was a matter of using a .zip compressed file to upload composer packages instead of a .tar.gz. I can't tell you why a tarball won't work, especially considering Artifactory's documentation explicitly mentions tar.gz, but it would appear .zip is the way to go.
When you first create a PHP composer repository in Artifactory it will not have the repository index yet (package.json file). The index will be created only after you deploy your first package.
It is possible that Artifactory was unable to read the package metadata for the package you deployed. In such a case, assuming this is the only package in the repository, the index will not be calculated.
In order to verify this is the case:
Browse the php-local repository in the Artifactory UI
Check for the presence of a .composer/packages.json file (relative to the repository root). If it does not exist, it is an indication that Artifactory was unable to calculate the repository index
Select the package you deployed, and switch to the Composer Info tab
Check that you can see the correct package version. If you see "No version found" - there was an issue with getting the version (probably because it is not included in the composer.json file)
Look for an error similar to the following in the artifactory log file:
Failed to persist package metadata for 'php-local/my-package.zip'.
Unable to read the package version from the extracted composer.json
file
If this is the case, you can solve the issue by:
Add a property named composer.version to your composer package and set its value to be the correct package version. You can do it by selecting the package file from the UI, switching to the Properties tab and adding a new property
If this done correctly you should be able to see the correct version in the Composer Info tab and that the packages.json file is being generated
If all of this does not help and the packages.json file is still not generated, there is probably another error which prevents Artifactory from generating it.
Try re-generating the repository index by:
Going to the root of the repository in the Artifactory UI
Right click the root of the repository and choose "Recalculate Index"
The index should be generated in the background
If the index is still not generated search the artifactory log file for any relevant error messages
Finally, one more option to try is deploying a different composer package and seeing if it behaves the same.
I have seen an Include / Exclude Pattern on the Jfrog repository that does not match the <vendor>/<package> naming standard cause this issue. We were able to upload artifacts but they were excluded metadata endpoint.
The following endpoint returned a 404 response code for every package until the Include / Exclude Pattern was fixed.
https://<org>.jfrog.io/artifactory/api/composer/<repo>/p2/<vendor>/<package>.json
I have a common (or not so coomon) scenario for yarn workspaces and didn't find the right guide for me online.
yarn workspaces look like that:
- monorepo
- packages
- client
- admin
- theme
- lib
Client is used as our endusers, it is a react project
Admin is used as backoffice for admin users and it is build in react too
Theme is used for all the UI kit (components) and storybook. We use the UI kit in client and admin
project and this is classic "monorepo style" (lerna) to share components between 2 projects. This folder should be shared only for this project.
Lib is used for all API and shared "Business logic" between multiple projects. I have 4 project which use the same lib functionality for API requests, Authentication, Redux and more.
Additional information:
monorepo is root repository with .gitmodules
Each sub folder is a different git repository
We use workspaces in order to have easy development on theme and on the client and admin project on the same time.
Questions:
We run yarn start only in client project and admin project. Both projects are using the same theme and same lib functionality. Because the lib is shared with other projects, it is updated on a weekly base:
How I can prevent from it to be updated from project to project? should I work with tags in git repository or should I remove lib from the monorepo worksapce and to work with it as npm package (the whole point is to have easy development process when we change the lib file we do not need to npm update it again and again.
If lib will be npm package, How can I tell monorepo to use workspaces when I run yarn start and to use the npm version when I run yarn build?
Please advice on the best practices for this scenario.
Thanks in advance,
Leo.
FINAL ANSWER:
I found the best solution for me and I tried it for 6 weeks during development (best practice).
I ended up with this structure:
monorepo // git MAIN repository
packages
client // git sub repository
admin // git sub repository
theme // git sub repository
lib // git sub repository
The client and admin use the theme as yarn workspaces https://classic.yarnpkg.com/en/docs/workspaces/
lib is used as Git npm package with git+ssh://git#gitlab.com:xxxx/xxx/lib.git#v1.0.1
The main/sub repositories structure gives me the ability the manage version control for each project separately and on the same time to use shared "theme" (workspaces) and "lib" core (npm) by versions.
Tip:
For easy development I recommend to add the lib as yarn workspace because when we run yarn start it hot reload the changes in real time. When we do yarn build we use the lib as npm package with an ssh link.
Good luck!
Leo.
Here is my personal preferences.
- monorepo
- packages
- client
- admin
- core
I think lib can be moved to core, and theme are more like npm package to me.
I recently added a new package and directory to my Go project, and now when I try to build it, I get errors about a password error on Gitlab.
I am not importing a new remote package, I am simply adding a new directory underneath my already declared module path. For instance, my go.mod has gitlab.com/example/api and the package I added is gitlab.com/example/api/postgres.
I am not actually hosting on gitlab, I just needed something to name the project as I worked on it. Clearly it won't find it on gitlab, but it is available locally. Why is go-get trying to download a package/path that is available locally?
Why is it only happening for this new package, and not for all of the existing package under this path?
Golang 1.14
You have to add replace above the require block in your go.mod to work with local package. For example:
replace gitlab.com/example => /Users/abc/projects/gitlab.com/example
Ref: https://github.com/golang/go/wiki/Modules
I'm trying to set up automated build for go projects. Most people just use github dependencies which don't need credentials. We have some internal dependencies however available on our private git central server. Credentials are needed however to have go access these.
A possible workaround would be to configure a global git variable inside our build machines / build dockers; something like:
git config --global url."https://user:password#private.git.server/".insteadOf "https://private.git.server/"
however this doesn't seem to be the best solution to me, since the password would be stored in a human-readable text file.
I think the git-credentials plugin should be able to help me out; could I maybe export GIT_TERMINAL_PROMPT=1 and let the git-credentials plugin fill in for me?
How could I make sure go get or go install gets access to our private repository in a secure way?
I use a workaround with GITHUB_TOKEN to solve this.
Generate GITHUB_TOKEN here https://github.com/settings/tokens
export GITHUB_TOKEN=xxx
git config --global url."https://${GITHUB_TOKEN}:x-oauth-basic#github.com/mycompany".insteadOf "https://github.com/mycompany"
This way you don't expose the password and can revoke token at any time.
Note: Go uses http when downloading dependencies, not ssh.
I am trying to create a module based on this example Oxid 6 module
https://github.com/linslin/oxid6-example-module
I have changed and cloned it to
https://github.com/v3ritas1989/gg_oxid6-example-module
as well as into my local module folder
..modules/compname/modulename
So for the beginning I want to change the modulename, submodul folder name and the namespace given
modules/companyname/modulename
Path Example:
modules/linslin/oxid6-example-module
However if I try to install my module
via composer
composer require compname/gg_oxid6-example-module:dev-master
(for local folder)
composer require v3ritas1989/gg_oxid6-example-module:dev-master
(for git)
I get ERROR:
[InvalidArgumentException] Could not find a matching version of
package compname/gg_oxid6-example-module. Check the package
spelling, your version constraint and that the package is available in
a stability which match es your minimum-stability (stable).
What I dont understand is why
composer require linslin/oxid6-example-module:dev-master
is working even so I deleted the local folder linslin
I guess it is pulling it from github?
but why then does this not work with my changed module and my github repo?
I made only search replace
modulename
compname
namespacename
Question:
I am assuming composer just does not find my repo or local.. so
How do I do composer require in a way that it only checks my local
folder?
Why does it also not find it in my github?
you need to register repository in your main composer.json file as described here:
https://getcomposer.org/doc/05-repositories.md#loading-a-package-from-a-vcs-repository
or register your package in https://packagist.org/ so it would be automatically detected.
There are also other ways of registering repositories, for example define it as local repository: https://getcomposer.org/doc/05-repositories.md#path