How to connect NPM to azure artifacts feed on Mac? - macos

I'm trying to connect to my private npm feed from Mac. I generated credentials from Connect to feed menu and they looked like that:
; begin auth token
//pkgs.dev.azure.com/<yourorganization>/_packaging/<yourfeed>/npm/registry/:username=ANYTHING-BUT-EMPTY
//pkgs.dev.azure.com/<yourorganization>/_packaging/<yourfeed>/npm/registry/:_password=BASE64-ENCODED-PAT-GOES-HERE
//pkgs.dev.azure.com/<yourorganization>/_packaging/<yourfeed>/npm/registry/:email=npm requires email to be set but doesn't use the value
//pkgs.dev.azure.com/<yourorganization>/_packaging/<yourfeed>/npm/:username=ANYTHING-BUT-EMPTY
//pkgs.dev.azure.com/<yourorganization>/_packaging/<yourfeed>/npm/:_password=BASE64-ENCODED-PAT-GOES-HERE
//pkgs.dev.azure.com/<yourorganization>/_packaging/<yourfeed>/npm/:email=npm requires email to be set but doesn't use the value
; end auth token
I placed that in .npmrc file in my project and it didn't work. When im trying to do npm install I get this error:
code E401
npm ERR! Unable to authenticate, need: Bearer authorization_uri=https://login.windows.net/...,
Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/", TFS-Federated
I also placed these credentials in $HOME directory which also didn't solve the issue. What am I doing wrong? In which .npmrc file should they be? Should I run additional commands to use them?

How to connect NPM to azure artifacts feed on Mac?
The .npmrc file which including the credentials should set int the $home directory.
Check the document Use npm to store JavaScript packages in Azure DevOps Services or TFS:
On your development machine, you also have an .npmrc file in $home for
Linux or Mac systems, or $env.HOME for Windows systems. This .npmrc
file should contain credentials for all of the registries that you
need to connect to. The npm client will look at your project's .npmrc
file, discover the registry, and fetch matching credentials from
$home/.npmrc or $env.HOME/.npmrc. The next section will discuss
credential acquisition.
Since it still not work for you, you could check if your npmrc token has expired. In your .npmrc, I found you are using BASE64-ENCODED-PAT-GOES-HERE, it seems you are using the PAT, but in the 90-day token type. The .npmrc file should like:
//pkgs.dev.azure.com/<yourorganization>/_packaging/<yourfeed>/npm/registry/:username=ANYTHING-BUT-EMPTY
//pkgs.dev.azure.com/<yourorganization>/_packaging/<yourfeed>/npm/registry/:_password=BASE64-ENCODED-PAT-GOES-HERE
//pkgs.dev.azure.com/<yourorganization>/_packaging/<yourfeed>/npm/registry/:email=YOUREMAIL#EXAMPLE.COM
//pkgs.dev.azure.com/<yourorganization>/_packaging/<yourfeed>/npm/registry/:always-auth=true
Check Create a token that lasts longer than 90 days.
If you still have 401 error, please check if your PAT has expired and convert to Base64String or do not have enough permission.
Hope this helps.

In my case for some reason I needed to surround the base 64 encoded token with double quotes and square bracket to make it work.
//pkgs.dev.azure.com/<yourorganization>/_packaging/<yourfeed>/npm/registry/:_password="[BASE64-ENCODED-PAT-GOES-HERE]"
Note the : "[ and at the end ]".
after adding that, all worked just fine.
I'm running MacOS Big Sur 11.6

To be able to connect to the ADO npm feed without saving the credentials, you can get an access token in ADO, and pass that as a parameter to the following script:
setup-npmrc-feed-auth.bash
#!/bin/bash
DecodedPat=$1
NPMAuthIdent=$(echo -ne "$DecodedPat" | base64);
pnpm config set registry https://pkgs.dev.azure.com/{orgName}/{projectName}/_packaging/{feedName}/npm/registry/ --location=global
pnpm config set //pkgs.dev.azure.com/{orgName}/{projectName}/_packaging/{feedName}/npm/registry/:username {orgName} --location=global
pnpm config set //pkgs.dev.azure.com/{orgName}/{projectName}/_packaging/{feedName}/npm/registry/:_password $NPMAuthIdent --location=global
pnpm config set //pkgs.dev.azure.com/{orgName}/{projectName}/_packaging/{feedName}/npm/registry/:email some#email.com --location=global
pnpm config set //pkgs.dev.azure.com/{orgName}/{projectName}/_packaging/{feedName}/npm/:username {orgName} --location=global
pnpm config set //pkgs.dev.azure.com/{orgName}/{projectName}/_packaging/{feedName}/npm/:_password $NPMAuthIdent --location=global
pnpm config set //pkgs.dev.azure.com/{orgName}/{projectName}/_packaging/{feedName}/npm/:email some#email.com --location=global
in your workspace, have the following:
.npmrc
registry=https://pkgs.dev.azure.com/{orgName}/{projectName}/_packaging/{feedName}/npm/registry/
auto-install-peers=true
strict-peer-dependencies=false
always-auth=true

Related

Goland modules with private gitlab (ssh)

Good day! I try to migrate from glide to go modules (private gitlab repos)
and checkout code via ssh
I have a simple project with an import from private gitlab repo.
go.mod looks like:
module my.private.package/modtest
go 1.12
require my.private.package/statistics v1.0.0
when I try to build my app or run test I get:
go: my.private.package/statistics#v1.0.0: unrecognized import path "my.private.package/statistics" (parse https://my.private.package/statistics?go-get=1: no go-import meta tags ())
go: error loading module requirements
I Tried to add settings to git config:
[url "ssh://git#my.gitlab.server:9999"]
insteadOf = https://my.private.package
But still getting this error.
Is there any way to make it work?
Thank you.
I've dealt with Go modules and a private GitLab before. Our private GitLab has groups and subgroups. The piece you are likely missing is ~/.netrc and you may have an improper global git configuration.
I've made a GitHub gist for this. You can find it here: https://gist.github.com/MicahParks/1ba2b19c39d1e5fccc3e892837b10e21
You can find the gist pasted below:
Problem
The go command line tool needs to be able to fetch dependencies from your private GitLab, but authenticaiton is required.
This assumes your private GitLab is hosted at privategitlab.company.com.
Environment variables
The following environment variables are recommended:
export GO111MODULE=on
export GOPRIVATE=privategitlab.company.com
The above lines might fit best in your shell startup, like a ~/.bashrc.
Explanation
GO111MODULE=on tells Golang command line tools you are using modules. I have not tested this with projects not using
Golang modules on a private GitLab.
GOPRIVATE=privategitlab.company.com tells Golang command line tools to not use public internet resources for the hostnames
listed (like the public module proxy).
Get a personal access token from your private GitLab
To future proof these instructions, please follow this guide from the GitLab docs.
I know that the read_api scope is required for Golang command line tools to work, and I may suspect read_repository as
well, but have not confirmed this.
Set up the ~/.netrc
In order for the Golang command line tools to authenticate to GitLab, a ~/.netrc file is best to use.
To create the file if it does not exist, run the following commands:
touch ~/.netrc
chmod 600 ~/.netrc
Now edit the contents of the file to match the following:
machine privategitlab.company.com login USERNAME_HERE password TOKEN_HERE
Where USERNAME_HERE is replaced with your GitLab username and TOKEN_HERE is replaced with the access token aquired in the
previous section.
Common mistakes
Do not set up a global git configuration with something along the lines of this:
git config --global url."git#privategitlab.company.com:".insteadOf "https://privategitlab.company.com"
I beleive at the time of writing this, the SSH git is not fully supported by Golang command line tools and this may cause
conflicts with the ~/.netrc.
Bonus: SSH config file
For regular use of the git tool, not the Golang command line tools, it's convient to have a ~/.ssh/config file set up.
In order to do this, run the following commands:
mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/config
chmod 600 ~/.ssh/config
Please note the permissions on the files and directory above are essentail for SSH to work in it's default configuration on
most Linux systems.
Then, edit the ~/.ssh/config file to match the following:
Host privategitlab.company.com
Hostname privategitlab.company.com
User USERNAME_HERE
IdentityFile ~/.ssh/id_rsa
Please note the spacing in the above file matters and will invalidate the file if it is incorrect.
Where USERNAME_HERE is your GitLab username and ~/.ssh/id_rsa is the path to your SSH private key in your file system.
You've already uploaded its public key to GitLab. Here are some instructions.
What version of Go are you using? If it's Go 1.13 or later, the default is to download modules through proxy.golang.org. You can change that for a particular set of packages using the GOPRIVATE environment variable.
Here is a quote from go help module-private, which I highly recommend reading in full. Once a module is fetched directly, it should use the same git/ssh logic as before modules.
The GOPRIVATE environment variable controls which modules the go command
considers to be private (not available publicly) and should therefore not use the
proxy or checksum database. The variable is a comma-separated list of
glob patterns (in the syntax of Go's path.Match) of module path prefixes.
For example,
GOPRIVATE=*.corp.example.com,rsc.io/private
causes the go command to treat as private any module with a path prefix
matching either pattern, including git.corp.example.com/xyzzy, rsc.io/private,
and rsc.io/private/quux.

How to access npm git-dependency from jenkins?

In my package.json I have a depedency to antoher git repository:
"my-dependeny":"git+https://somehost.com/my-dependency.git#<commit-hash>"
After running npm install this works fine while developing because git is using my personal stored credentials.
But executing npm install in a jenkins-pipline will fail, because there are not stored credentials. So I first tried to generate a deploy token in gitlab and edit the package.json:
"my-dependeny":"git+https://<deploy-token>:<password>#somehost.com/my-dependency.git#<commit-hash>"
Now it will work on jenkins!
But the problem with this approach is, that Windows autoamtically updates the credential for the host (somehost.com) in the Credentials Manager:
So when any developer is running npm install and is afterwards trying to access another project within the same host it will fail, because it will use the deploy-token for the autenthication.
Of course you could add new user to gitlab which has access to all relevant projects. Then you could store this user on jenkins, but this doesn't seem to be an optimal or even good solution.
Maybe there could also be a way to stop the Credential Manager from updating git credentials. But this also doesn't sound well, because every developer would have to prevent this, which is not very intuitive.
My Question
Is there a good way to access npm git-depencies from jenkins without requiring extra steps that need to be done by each developer?
I was finally able to find a solution by myself with the help of git-credential-store.
The following steps apply for linux systems, because in my case the Jenkins is running on Linux. But there should be a similiar solution for Windows systems.
Create empty ~/.git-credentials file. If you use a Dockerfile to copy the .git-credentials file you most likly have to use /root/git-credentials as path (e.g. COPY .git-credentials /root/)
Insert your credentials like the following to your ~/.git-credentials file:
<protocol>://<deploy-token>:<password>#<hostname>/<project>
Example:
https://gitlab+deploy-10:mYSecReTPaSSWord#somehost.com/my-dependency.git
or multiple credentials (follow 4. step below to enable this feature):
https://gitlab+deploy-10:mYSecReTPaSSWord#somehost.de/some-path/some-project.git
https://gitlab+deploy-11:otHerPaSSWord#somehost.de/some-path/some-other-project.git
NOTE:
The line-endings in the ~/.git-credentials have to be LF. If
they are CR;LF unexpected errors can occur.
git config --global credential.helper store
This will tell git that you wanna use your .git-credentials file for authentication.
(~/.git-credentials is the default location)
optional: git config --global credential.useHttpPath true
This is needed if you want to use mutliple Deploy Tokens for multiple projects within the same hostname. Without credential.useHttpPath true in the example above this would mean, that gitlab+deploy-10:mYSecReTPaSSWord would be used in every request to somehost.de. The HTTP path would be ignored completly.
The following is how I solved this in my jenkins file:
withCredentials([usernamePassword(credentialsId: 'my-git-username-and-api-token', passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')]) {
sh '''
git config --global credential.username ${GIT_USERNAME}
git config --global credential.helper "!echo password=${GIT_PASSWORD}; echo"
npm install
npm run build
'''
}
As you can see it uses GitHub username and API token from the credentials store. It also runs git config every time. For me this is necessary as I'm running my builds in a docker node. This solves the problem we encountered where git+https dependencies in npm were not being resolved in our jenkins pipeline.

set email verification on bitnami Parse server from android studio

I have been searching day and night for a way to set email verification for my app with its server on Bitnami Parse Server(EC2). To no avail! All I could find was a client asking me to shift my app to back4App. But to switch my app to back4App only because I have no solution to email verification is ridiculous which is why i am stalling. Help Please!
I am looking for a tutorial or some detailed guide. I am not at all sure on how to download parse server mailgun.
Did you enable HTTPS? Could you check this post? It might help you setting it up:
https://community.bitnami.com/t/parse-email-verification-issue/53354/
Access your server via Putty.
In the terminal go to directory htdocs by using command
$cd /opt/bitnami/apps/parse/htdocs.
Here you will find one file package.json.
Open the file using vi and see whether it contains something or not
If it is empty then add {} to your package.json file
save it and run
$npm install #parse/simple-mailgun-adapter --save
After this open your server.js file set up your emailAdapter with appName and publicserverurl.
After this add this line to your server.js file
verifyUserEmails: true

Imagemagick - change policy.xml on Heroku

I'm trying to access images via https on Heroku with Imagemagick. How can I change the policies (in policy.xml) on Heroku?
Heroku made an "ImageMagick security update" in May, 2016: https://devcenter.heroku.com/changelog-items/891
I can see the policy list, after typing heroku run bash and convert -list policy:
Path: [built-in]
Policy: Undefined
rights: None
Path: /etc/ImageMagick/policy.xml
[...]
Policy: Coder
rights: None
pattern: HTTPS
[...]
How can I change the policy?
update 1: this is the error in the log file:
Command failed: convert.im6: not authorized `//scontent-fra3-1.xx.fbcdn.net/v/t1.0-9/13962741_132344500547278_4974691444630710043_n.jpg?oh=c169b4ffce9e5ce330ee99214cc6b8d5&oe=5880F245'
I’ve found a relatively simple solution.
Create a .magick directory in your app’s source, and add your policy.xml there. Then, you’ll have to set the environment variable MAGICK_CONFIGURE_PATH to /app/.magick in order to load your file with higher precedence than the default one.
We need to install the third party software ImageMagick on heroku. I used this https://github.com/ello/heroku-buildpack-imagemagick build pack for installing ImageMagick.
So, inside bin/compile, there is a policy file, which is restricting the images to read over Https, enable the attribute rights to read which allows to read over Https
Fork the repo and do your changes, commit and add that repository url to your heroku buildpacks
Read the warnings at ImageTragick, then make a backup and delete the line that restricts you.
You can find the file to edit in the same directory as the other XML config files by doing the following - the file is called policy.xml:
convert -debug configure -list font 2>&1 | grep -E "Searching|Loading"

How to exclude certain domains from an npm proxy

We use npm behind a company firewall and thus use proxy and https-proxy settings in the npm configuration. This works fine as long as all npm modules are loaded from an external registry. But as a matter of fact we also have internal modules on an internal github. The access to these modules must not use the proxy of course. My question is: Can I specify a list of domains in the npm configuration, for which the proxy should not be used? Something like the no_proxy environment variable in the unix shell?
Assuming your environment looks like this:
Build server with internet access only over proxy: your.proxy.host:3128
Local Nexus Registry: https://your.local.nexus.registry/nexus/content/groups/npm/
NPM must use local Nexus Registry. Configuration file: .npmrc
registry = https://your.local.nexus.registry/nexus/content/groups/npm/
You can tell npm to use a proxy by setting the environment variables
http_proxy=http://your.proxy.host:3128
https_proxy=http://your.proxy.host:3128
but then npm will also try to reach your (local) Nexus Registry using the proxy.
You need to have one of the latest npm Versions (npm 2.14.7 works fine) and set an additional environment variable to exclude your Nexus Registry from the proxy:
no_proxy=your.local.nexus.registry
Since NPM 6.4.1, released on 2018-08-22, you can use the noproxy option, even with a custom registry configured.
Example :
npm config set registry "http://<my-npm-registry-host>:<registry-port>"
npm config set proxy "http://<my-proxy-host>:<proxy-port>"
npm config set https-proxy "http://<my-proxy-host>:<proxy-port>"
npm config set noproxy "my-proxy-host" (accepts pattern like *.domain)
Check config :
npm config list
References :
NPM doc for noproxy
NPM changelog
NPM PR #46
to exclude certain domains , you can add below lines in .npmrc
noproxy[]=.local.nexus
noproxy[]=.internal.npm.repo
https://docs.npmjs.com/misc/config#noproxy
As adiesner indicated, you cannot set a noproxy/no_proxy in the npm config, so the only way to do this is via the environment variables:
npm config set registry "your.local.nexus.registry/nexus/content/groups/npm/"
export http_proxy="http://your.proxy.host:3128"
export https_proxy=$http_proxy
export no_proxy="your.local.nexus.registry"
If you are using cntlm you should configure the NoProxy option in the configuration file "/etc/cntlm.conf". For instance,
Username your_user
Domain UCI.CU
Password your_password
Proxy 10.0.0.1:8080
Listen 3128
NoProxy uci.cu, edu.cu, nexus.prod.uci.cu, localhost, 127.0.0.*, 10.*, 192.168.*
Remember that you must restart cntlm: sudo systemctl restart cntlm
For PowerShell, the commands are:
$Env:http_proxy = "http://your.proxy.host:3128"
$Env:https_proxy = "http://your.proxy.host:3128"
$Env:no_proxy = "..."

Resources