File not found in Gitlab-CI - continuous-integration

I've already seen this question/answer here, but the solution isn't working for a Flutter project I'm working on.
Here's my yaml file:
image: cirrusci/flutter
variables:
before_script:
- flutter channel beta
- flutter upgrade
stages:
# - build
- test
#build:
# stage: build
# script:
# - flutter build apk
unitTests:
stage: test
script:
# - ls -la /opt/application/
# - ls -la ~/
- bash -c "echo \"$APP_VARS\" > ./cfg/env.json"
- flutter test test/widget_test.dart
Which yielded a cannot access <filename/directory>: File or Directory Not Found error with both ls -la /opt/application/ and bash -c "echo \"$APP_VARS\" > ./cfg/env.json"
Her's the output from ls -la ~/:
$ ls -la ~/
total 72
drwxr-xr-x. 1 cirrus cirrus 4096 Jul 9 19:16 .
drwxr-xr-x. 1 root root 4096 Mar 7 2018 ..
drwxr-xr-x. 1 cirrus cirrus 4096 Jul 9 19:17 .android
-rw-r--r--. 1 cirrus cirrus 220 May 15 2017 .bash_logout
-rw-r--r--. 1 cirrus cirrus 3526 May 15 2017 .bashrc
-rw-r--r--. 1 cirrus cirrus 24 Jul 9 19:16 .flutter
-rw-r--r--. 1 cirrus cirrus 675 May 15 2017 .profile
drwxr-xr-x. 1 cirrus cirrus 4096 Jul 9 19:16 .pub-cache
drwxr-xr-x. 1 cirrus cirrus 4096 Jul 9 19:16 sdks
My intent is to inject environment variables via the CI for use during testing (and hopefully deployment, although I haven't gotten that far yet).

The solution is two-part:
First, the /builds/**/cfg/ directory actually didn't exist. To fix this, you have two options-- do a mkdir or commit/push a file in the repo. I added an empty .gitignore to my cfg directory.
Then, that yielded a Permission Denied error. To solve this, I first tried to sudo my commands (which let me perform the mkdir mentioned above). When that didn't work, I did a chmod -R 777 /builds/, and that fixed my issue.
Going forward, I plan to use chmod [-R] 644 or chmod [-R] 755 instead and do a chmod +x on any files I need to run via a shell script.
I also moved my environment configuration script to the before_script: section in the gitlab YAML, but I don't believe that was necessary.
See for info on chmod.

Related

Having trouble with declared functions in `~/.profile` not being loaded in integrated terminal shells

I have a .profile which is copied to /home/vscode during my container build before running common-debian.sh
This is important as during container image build common-delian.sh will skip creating a default .profile if the file is already present
In that profile I have a number of bash functions and some exported variables.
One of the variables relies on executing one of the declared functions, and in my shell I can see the correct variable and value is set.
This problem arises when opening a new integrated terminal in vscode.
"new terminal" command
However none of my functions are available in the shell to execute.
Confirmed through typeset -F
I can run source ~/.profile and the functions are then available in the shell.
Is there some place I can turn up logging to determine what is happening here?
Details about the install
vscode Host install is Mac M1
VERSION='0.202.6'
CONTENTS_URL='https://github.com/microsoft/vscode-dev-containers/tree/main/containers/debian/history/0.202.6.md'
DEFINITION_ID='debian'
VARIANT='bullseye'
GIT_REPOSITORY='https://github.com/microsoft/vscode-dev-containers/'
GIT_REPOSITORY_RELEASE='v0.224.0'
BUILD_TIMESTAMP='Fri, 25 Feb 2022 10:48:36 GMT'
Example function declaration
kconfig() {
ls $HOME/.kube/configs/* | tr '\n' ':'
}
declare -f kconfig
Example variable making use
export KUBECONFIG="$(kconfig)"
I can confirm before sourcing .profile, this variable is set and valid and should only have come from the .profile declaration.
home folder for reference
vscode ➜ /workspace (master ✗) $ ls -al ~
total 76
drwxr-xr-x 1 vscode vscode 4096 Apr 27 18:59 .
drwxr-xr-x 1 root root 4096 Feb 25 10:55 ..
drwxr-xr-x 1 501 dialout 160 Mar 8 22:31 .aws
-rw-r--r-- 1 vscode vscode 220 Aug 4 2021 .bash_logout
-rw-r--r-- 1 vscode vscode 4665 Feb 25 10:55 .bashrc
drwxr-xr-x 3 vscode vscode 4096 Apr 27 18:59 .cache
drwxr-xr-x 1 501 dialout 192 Apr 25 21:41 .config
drwxr-xr-x 1 501 dialout 160 Apr 27 14:20 .creds
-rw-r--r-- 1 vscode vscode 500 Apr 27 18:59 .gitconfig
drwx------ 2 vscode vscode 4096 Apr 27 18:59 .gnupg
drwxr-xr-x 1 501 dialout 160 Mar 9 21:59 .kube
drwxr-xr-x 12 vscode vscode 4096 Feb 25 10:55 .oh-my-zsh
-rw-r--r-- 1 vscode vscode 2381 Apr 27 19:19 .profile
drwxr-xr-x 1 vscode root 4096 Apr 27 18:59 .vscode-server
drwxr-xr-x 3 vscode root 4096 Apr 27 18:26 .vscode-server-insiders
-rw-r--r-- 1 vscode vscode 3897 Feb 25 10:55 .zshrc
Edit below:
Now I am copying this file in as ~/.bash_profile
There is still a .profile, vscode common-debian.sh creates this if the file is not present when building the container image.
vscode ➜ /workspace (master ✗) $ ls -al ~
total 80
drwxr-xr-x 1 vscode vscode 4096 Apr 27 20:08 .
drwxr-xr-x 1 root root 4096 Feb 25 10:55 ..
drwxr-xr-x 1 501 dialout 160 Mar 8 22:31 .aws
-rw-r--r-- 1 vscode vscode 220 Aug 4 2021 .bash_logout
-rw-r--r-- 1 vscode vscode 2671 Apr 27 19:57 .bash_profile
-rw-r--r-- 1 vscode vscode 4665 Feb 25 10:55 .bashrc
drwxr-xr-x 3 vscode vscode 4096 Apr 27 20:08 .cache
drwxr-xr-x 1 501 dialout 192 Apr 25 21:41 .config
drwxr-xr-x 1 501 dialout 160 Apr 27 14:20 .creds
-rw-r--r-- 1 vscode vscode 500 Apr 27 20:08 .gitconfig
drwx------ 2 vscode vscode 4096 Apr 27 20:08 .gnupg
drwxr-xr-x 1 501 dialout 160 Mar 9 21:59 .kube
drwxr-xr-x 12 vscode vscode 4096 Feb 25 10:55 .oh-my-zsh
-rw-r--r-- 1 vscode vscode 807 Aug 4 2021 .profile
drwxr-xr-x 1 vscode root 4096 Apr 27 20:08 .vscode-server
drwxr-xr-x 3 vscode root 4096 Apr 27 20:07 .vscode-server-insiders
-rw-r--r-- 1 vscode vscode 3897 Feb 25 10:55 .zshrc
also added the following to /etc/profile.d/
# This file is intended to be copied into /etc/profile.d
project_profile="$HOME/.bash_profile"
if [ -r $project_profile ]; then
source $project_profile
else
echo "File not found or not readable: $project_profile"
fi
none of which has had an effect on the resulting new shells
perhaps ignoring vscode
docker exec -it {container} /bin/bash still has the same results.
I have also tried creating an empty terminal profile with an empty shell profile, launching vscode from that terminal window.
Thinking that perhaps inherited env was causing issues.
Also set the following user setting
"terminal.integrated.inheritEnv": false,
From the man page for bash (in the INVOCATION section):
When bash is started non-interactively, to run a shell script, for
example, it looks for the variable BASH_ENV in the environment, expands
its value if it appears there, and uses the expanded value as the name
of a file to read and execute. Bash behaves as if the following command
were executed:
if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
but the value of the PATH variable is not used to search for the filename.
In your use case, i.e. a shell script, no startup file is read. But you can change that by setting the value of BASH_ENV to a file you want to read on startup. Alternately your script can source anything you like when it runs, e.g. source .profile.
Extracted from answer comments:
Original Poster noted that the issue was fixed using this in vscode:
"terminal.integrated.defaultProfile.linux": "bash",
"terminal.integrated.profiles.linux": {
"bash": {
"path": "/bin/bash",
"icon": "terminal-bash",
"args": [ "-l" ]
},
}
(The -l arg to make it a login shell being, probably, the important part?)

.jupyter cannot be deleted or written to

My jupyter notebook command stopped working, and I believe it is because my .jupyter directory is behaving strangely.
If I try to delete it, it neither errors or gets deleted.
For example, this works, but should not.
rm -rf .jupyter
cd .jupyter
If I then run,
touch hello.txt
I get touch: hello.txt: Invalid argument
Has anyone run into this and solved it?
More Info
I don't think it's a symlink:
(py36) ➜ ~ ls -halt .jupyter
total 0
drwxr-xr-x+ 389 shleifer staff 13K Jul 5 19:25 ../
drwxr-xr-x 10 shleifer staff 320 Dec 28 2019 ./

Dockerfile failed to change user access

I want to run jenkins on docker and change the user access so could read the SSH key and access git.
Here is sample of the dockerfile
FROM jenkins/jenkins:lts
USER root
COPY --chown=jenkins:jenkins id_rsa $JENKINS_HOME/.ssh/id_rsa
COPY --chown=jenkins:jenkins id_rsa.pub $JENKINS_HOME/.ssh/id_rsa.pub
RUN /bin/bash -c 'ls -la $JENKINS_HOME/.ssh; chmod 600 -R $JENKINS_HOME/.ssh; ls -la $JENKINS_HOME/.ssh'
The output upon build is a success, access has been changed!
Step 3/3 : RUN /bin/bash -c 'ls -la $JENKINS_HOME/.ssh; chmod 600 -R $JENKINS_HOME/.ssh; ls -la $JENKINS_HOME/.ssh'
---> Running in 137d1a4f9f6d
total 16
drwxr-xr-x 2 jenkins jenkins 4096 Jan 8 04:11 .
drwxr-xr-x 3 jenkins jenkins 4096 Jan 8 04:11 ..
-rwxr-xr-x 1 jenkins jenkins 1843 Jan 2 02:33 id_rsa
-rwxr-xr-x 1 jenkins jenkins 413 Jan 2 02:33 id_rsa.pub
total 16
drw------- 2 jenkins jenkins 4096 Jan 8 04:11 .
drwxr-xr-x 3 jenkins jenkins 4096 Jan 8 04:11 ..
-rw------- 1 jenkins jenkins 1843 Jan 2 02:33 id_rsa
-rw------- 1 jenkins jenkins 413 Jan 2 02:33 id_rsa.pub
Removing intermediate container 137d1a4f9f6d
---> 7d6334d2b044
However when I go inside the /bin/bash the access is set to default, the chmod was not working
jenkins#f49048ec8c88:/$ ls -al /var/jenkins_home/.ssh/
total 16
drwxr-xr-x 2 jenkins jenkins 4096 Jan 8 04:25 .
drwxr-xr-x 16 jenkins jenkins 4096 Jan 8 04:26 ..
-rwxr-xr-x 1 jenkins jenkins 1843 Jan 2 02:33 id_rsa
-rwxr-xr-x 1 jenkins jenkins 413 Jan 2 02:33 id_rsa.pub
any idea why the behavior is this way?
This happened because $JENKINS_HOME is defined as VOLUME in jenkins:lts base image. You can workaround this in any of the 3 ways
You can fix the permissions on host machine before building and it should work.
You can use multi stage build, change the permission and copy files from first stage
FROM jenkins/jenkins:lts as base
USER root
COPY --chown=jenkins:jenkins id_rsa /tmp/ssh_keys/
COPY --chown=jenkins:jenkins id_rsa.pub /tmp/ssh_keys/
RUN chmod 600 -R /tmp/ssh_keys
FROM jenkins/jenkins:lts
USER root
COPY --chown=jenkins:jenkins --from=base /tmp/ssh_keys $JENKINS_HOME/.ssh
You can copy and change permissions of the files in a temporary directory as part of build. As part of your startup script(entrypoint), you can copy them from the temporary directory to actual directory.
Because the command /bin/bash is just for that session. If you want it to be permanent, put it inside .profile.

Messed up my bash_profile and bashrc files - reset to default?

I think I messed up my .bash_profile and .bashrc file.
Is there any way I can reset them to default?
When I want to access them through the terminal I get: Permission denied.
If I open them using sublime text, my .bashrc is completely empty and my .bash_profile contains the following code:
PATH="/Library/Frameworks/Python.framework/Versions/3.8/bin:${PATH}"
export PATH=/usr/local/bin:$PATH
alias python=python3
alias sublime="open -a /Applications/Sublime\ Text.app"
source ~/.profile
Any idea on what I could add/delete to any of the files?
Python does not run properly anymore since I accidentally played around on those files.
You may find the default .bash_profile and .bashrc files in /etc/skel/
[user#server /]$ ls -la /etc/skel
total 28
drwxr-xr-x. 3 root root 78 Jan 3 2019 .
drwxr-xr-x. 131 root root 12288 Nov 7 13:03 ..
-rw-r--r--. 1 root root 18 Oct 30 2018 .bash_logout
-rw-r--r--. 1 root root 193 Oct 30 2018 .bash_profile
-rw-r--r--. 1 root root 231 Oct 30 2018 .bashrc
If it's the case you can get them back:
cp /etc/skel/.bash_profile /etc/skel/.bashrc ~/
Make a backup of your current files before ;)
Just to explain: the content of /etc/skel/ is copied into the home directory of a user when created with adduser.

-bash: cd: app: No such file or directory

I simply changed the name in finder and now when I use a command in terminal, this is the error I keep receiving. I've tried to look at all the other cases of the same error but they all pertain to different solutions that don't work. The path is clearly correct because when I try to cd into the directory, I'm already in the directory it is contained in.
Johns-MacBook-Air:SongAnalysisSentiment johnanukem$ cd app
-bash: cd: app: No such file or directory
Johns-MacBook-Air:SongAnalysisSentiment johnanukem$ ls
README app sentiments.csv songdictionary.py
Johns-MacBook-Air:SongAnalysisSentiment johnanukem$ ls -lah
total 776
drwxr-xr-x 8 johnanukem staff 272B Nov 14 02:38 .
drwx------+ 39 johnanukem staff 1.3K Nov 14 02:38 ..
-rw-r--r--# 1 johnanukem staff 6.0K Nov 14 01:39 .DS_Store
drwxr-xr-x 16 johnanukem staff 544B Nov 14 02:43 .git
-rw-r--r-- 1 johnanukem staff 35B Nov 14 01:38 README
drwxr-xr-x 8 johnanukem staff 272B Nov 14 02:38 app
-rw-r--r-- 1 johnanukem staff 370K Nov 14 01:49 sentiments.csv
-rw-r--r-- 1 johnanukem staff 1.3K Nov 14 02:29 songdictionary.py
type cd ap and then [Press Tab] to autocomplete, may be there is some spaces after app
On MacOS with the default Bash
$ bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin17)
Copyright (C) 2007 Free Software Foundation, Inc.
cd app raises the error
-bash: cd: app: No such file or directory
due to a bug of the super old Bash version installed with MacOS.
The workaround is to prefix the directory with ./ in this way
cd ./app
and it will work.
The alternative is to install a modern and better Bash version using Homebrew.
brew install bash

Resources