Actually, I have my zip files on gitlab, I want to extract those files using gitlab CI/CD.I have tried this in .gitlab-ci.yml:
image: docker
stages:
- build
- test
services:
- docker:dind
build:
before_script:
- apk add p7zip
script:
- cd \kmfs
- 7z x -oChassisA ChassisA.zip
OUTPUT
$ apk add p7zip
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
(1/3) Installing libgcc (6.4.0-r9)
(2/3) Installing libstdc++ (6.4.0-r9)
(3/3) Installing p7zip (16.02-r3)
Executing busybox-1.28.4-r3.trigger
OK: 11 MiB in 17 packages
$ cd \kmfs
$ 7z x -oChassisA ChassisA.zip
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C.UTF-8,Utf16=on,HugeFiles=on,64 bits,1 CPU Intel(R) Xeon(R) CPU # 2.30GHz (306F0),ASM,AES-NI)
Scanning the drive for archives:
1 file, 3638943 bytes (3554 KiB)
Extracting archive: ChassisA.zip
--
Path = ChassisA.zip
Type = zip
Physical Size = 3638943
Everything is Ok
Folders: 1
Files: 4
Size: 24070952
Compressed: 3638943
Job succeeded
It is executing successfully but extracted files are not reflected in gitlab repositories and I am not able to access those files using node js code written in test stage.
So it will be great if someone suggests me any way to extract the .zip files on gitlab itself either by using commands or from nodejs code...
You need to use artifacts as part of your job to declared the path of the extracted zip. This will allow the subsequent jobs to access those files.
artifacts:
when: always
expire_in: 1 week
paths:
- zip files folder structure after its extracted
Related
I want to build an android signed APK and receive release APK through the slack channel. Tried the below script but it's not working due to my app written with JDK 8.
This is the script which I used.
image: jangrewe/gitlab-ci-android
cache:
key: ${CI_PROJECT_ID}
paths:
- .gradle/
before_script:
- export GRADLE_USER_HOME=$(pwd)/.gradle
- chmod +x ./gradlew
stages:
- build
assembleDebug:
stage: build
only:
- development
- tags
script:
- ./gradlew assembleDebug
- |
curl \
-F token="${SLACK_CHANNEL_ACCESS_TOKEN}" \
-F channels="${SLACK_CHANNEL_ID}" \
-F initial_comment="Hello team! Here is the latest APK" \
-F "file=#$(find app/build/outputs/apk/debug -name 'MyApp*')" \
https://slack.com/api/files.upload
artifacts:
paths:
- app/build/outputs/apk/debug
view raw
But is showing some java classes not found. (That java files deprecated in Java 11)
First, you need to setup slack authentication keys.
Create App in Slack
Go to Authentication Section and Generate Authentication Key.
Get Channel Id which you want to receive messages.
Mention your app name in your slack thread and add the app to the channel.
Setup those keys in your GitLab ci setting variables.
SLACK_CHANNEL_ACCESS_TOKEN = Access Token Generated by Slack App
SLACK_CHANNEL_ID = Channel Id (Check URL Last section for the channel id)
8.Copy your existing Keystore file to the repository. (Please do this if your project is private.)
7.Change GitLab script's content to the below code.
Make sure to change certificate password,key password and alias.
image: openjdk:8-jdk
variables:
# ANDROID_COMPILE_SDK is the version of Android you're compiling with.
# It should match compileSdkVersion.
ANDROID_COMPILE_SDK: "29"
# ANDROID_BUILD_TOOLS is the version of the Android build tools you are using.
# It should match buildToolsVersion.
ANDROID_BUILD_TOOLS: "29.0.3"
# It's what version of the command line tools we're going to download from the official site.
# Official Site-> https://developer.android.com/studio/index.html
# There, look down below at the cli tools only, sdk tools package is of format:
# commandlinetools-os_type-ANDROID_SDK_TOOLS_latest.zip
# when the script was last modified for latest compileSdkVersion, it was which is written down below
ANDROID_SDK_TOOLS: "6514223"
# Packages installation before running script
before_script:
- apt-get --quiet update --yes
- apt-get --quiet install --yes wget tar unzip lib32stdc++6 lib32z1
# Setup path as android_home for moving/exporting the downloaded sdk into it
- export ANDROID_HOME="${PWD}/android-home"
# Create a new directory at specified location
- install -d $ANDROID_HOME
# Here we are installing androidSDK tools from official source,
# (the key thing here is the url from where you are downloading these sdk tool for command line, so please do note this url pattern there and here as well)
# after that unzipping those tools and
# then running a series of SDK manager commands to install necessary android SDK packages that'll allow the app to build
- wget --output-document=$ANDROID_HOME/cmdline-tools.zip https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_SDK_TOOLS}_latest.zip
# move to the archive at ANDROID_HOME
- pushd $ANDROID_HOME
- unzip -d cmdline-tools cmdline-tools.zip
- popd
- export PATH=$PATH:${ANDROID_HOME}/cmdline-tools/tools/bin/
# Nothing fancy here, just checking sdkManager version
- sdkmanager --version
# use yes to accept all licenses
- yes | sdkmanager --sdk_root=${ANDROID_HOME} --licenses || true
- sdkmanager --sdk_root=${ANDROID_HOME} "platforms;android-${ANDROID_COMPILE_SDK}"
- sdkmanager --sdk_root=${ANDROID_HOME} "platform-tools"
- sdkmanager --sdk_root=${ANDROID_HOME} "build-tools;${ANDROID_BUILD_TOOLS}"
# Not necessary, but just for surity
- chmod +x ./gradlew
# Make Project
assembleDebug:
interruptible: true
stage: build
only:
- tags
script:
- ls
- last_v=$(git describe --abbrev=0 2>/dev/null || echo '')
- tag_message=$(git tag -l -n9 $last_v)
- echo $last_v
- echo $tag_message
- ./gradlew assembleRelease
-Pandroid.injected.signing.store.file=$(pwd)/Certificate.jks
-Pandroid.injected.signing.store.password=123456
-Pandroid.injected.signing.key.alias=key0
-Pandroid.injected.signing.key.password=123456
- |
curl \
-F token="${SLACK_CHANNEL_ACCESS_TOKEN}" \
-F channels="${SLACK_CHANNEL_ID}" \
-F initial_comment="$tag_message" \
-F "file=#$(find app/build/outputs/apk/release -name 'app*')" \
https://slack.com/api/files.upload
artifacts:
paths:
- app/build/outputs/
Im try so setup a CI/CD Pipeline for a Nativescript app, added the commands to install node and npm install but nativescript has dependencies that it need. How do I go about on Azure dev ops dynamically without having to create a vm that has nativescript and all its dependencies installed and setup
So i have used a VM and install nativescript on it and used and agent to connect to the machine and build solution, i have done the same using jenkins but jenkins was running on the vm, no i want to move the whole pipeline on azure dev ops
command used in build step: tns build android
If you don't want to use a vm you'll have to install everything needed for nativescript before building it on their hosted agent each time you create a build for your app.
A couple of important things to note. First the name of your repository is changed to 's' this messes with the naming of your entitlement file... or at least it did for me. I fix this with a bash file I add to my repository that changes the name of the path in build.xcconfig for the CODE_SIGN_ENTITLEMENTS variable. I added an npm run entitle command in my package.json file to run this before building. Second you'll want to store all files and secure passwords in the library section under pipelines in you Azure Devops project. Third, Using the classic editor is your best friend for figuring out yaml as most jobs have an option to view the YAML. You can also use the classic editor as an alternative to the YAML file
The YAML and bash file below show an example of how you can build an ipa and apk file that is stored as an artifact. You can then use that trigger a release pipeline to push the the play and app store.
# YAML File
name: Release Build
trigger:
- release/* # will start build for pull request into release branch ie. realease/version_1_0_0, release/version_2_0_0
pool:
vmImage: 'macOS-10.13'
variables:
scheme: 's' # default name/scheme created on this machine for ipa
sdk: 'iphoneos'
configuration: 'Release'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.14'
displayName: 'Install Node.js'
# Download Secure File for Android
# Note: if multiple secure files are downloaded... variable name will change and break pipeline
- task: DownloadSecureFile#1
displayName: 'download android keystore file'
inputs:
secureFile: myKeystore.keystore
#Install Apple Certificate(Distrobution)
- task: InstallAppleCertificate#2
displayName: 'Install an Apple certificate Distribution (yourcertificate.p12)'
inputs:
certSecureFile: '00000000-0000-0000-0000-000000000000' # got id from viewing file in clasic editor for pipeline
certPwd: '$(myCertificatePasswordP12)' # password stored in Library
# Install Apple Provisioning Profile(Distrobution)
- task: InstallAppleProvisioningProfile#1
displayName: 'Apple Provisioning Profile(myProvisioningProfile.mobileprovision)'
inputs:
provisioningProfileLocation: 'secureFiles' # Options: secureFiles, sourceRepository
provProfileSecureFile: '00000000-0000-0000-0000-000000000000' # Required when provisioningProfileLocation == SecureFiles
# General Set Up
- script: |
npm install -g nativescript#latest
npm install
displayName: 'Install native script and node Modules'
# variable explination
# $DOWNLOADSECUREFILE_SECUREFILEPATH is keystore file downloaded earlier
# $KEYSTORE_PASSWORD refers to the environment variable in this script which references library variable
# $(MyPlayStoreAlias) refers to library variable for your apps alias
# $BUILD_SOURCESDIRECTORY location where apk is built to
# Android
- script: |
tns build android --env.production --release --key-store-path $DOWNLOADSECUREFILE_SECUREFILEPATH --key-store-password $KEYSTORE_PASSWORD --key-store-alias $(MyPlayStoreAlias) --key-store-alias-password $KEYSTORE_PASSWORD --bundle --copy-to $BUILD_SOURCESDIRECTORY #creates apk
displayName: 'Build Android Release apk'
env:
KEYSTORE_PASSWORD: $(MyPlayStoreKeystore)
# create apk artifact
- task: PublishBuildArtifacts#1
inputs:
pathtoPublish: '$(Build.SourcesDirectory)/app-release.apk'
artifactName: 'apkDrop'
displayName: 'Publishing apkDrop artifact'
# have to use xcode 10.1 to meet min standards for uploading ipa... default version for this machine was lower than 10.1
#changing xcode version
- script: |
xcodebuild -version
/bin/bash -c "echo '##vso[task.setvariable variable=MD_APPLE_SDK_ROOT;]'/Applications/Xcode_10.1.app;sudo xcode-select --switch /Applications/Xcode_10.1.app/Contents/Developer"
xcodebuild -version
displayName: 'changing xcode to 10.1'
# Optional... was running into build issues with latest version
#downgrading cocoapods version
- script: |
sudo gem uninstall cocoapods
sudo gem install cocoapods -v 1.5.3
displayName: 'Using cocoapods version 1.5.3'
#iOS
- script: |
xcodebuild -version # makeing sure the correct xcode version is being used
pip install --ignore-installed six # fixes pip 6 error
npm run entitle #custom bash script used to change entitlement file
tns run ios --provision #see what provisioning profile and certificate are installed... helpful for debugging
tns build ios --env.production --release --bundle #creates xcworkspace
displayName: 'Build ios Release xcworkspace'
#build and sign ipa
- task: Xcode#5
displayName: 'Xcode sign and build'
inputs:
sdk: '$(sdk)' # custom var
scheme: '$(scheme)' # must be provided if setting manual path to xcworkspace
configuration: '$(configuration)' # custom var
xcodeVersion: 'specifyPath'
xcodeDeveloperDir: '/Applications/Xcode_10.1.app' #using xcode 10.1
xcWorkspacePath: 'platforms/ios/s.xcworkspace'
exportPath: '$(agent.buildDirectory)/output/$(sdk)/$(configuration)' #location where ipa file will be stored
packageApp: true #create ipa
signingOption: manual
signingIdentity: '$(APPLE_CERTIFICATE_SIGNING_IDENTITY)' # distribution certificate
provisioningProfileUuid: '$(APPLE_PROV_PROFILE_UUID)' # distribution profile
#creating ipa artifact
- task: PublishBuildArtifacts#1
displayName: 'Publishing ipaDrop artifact'
inputs:
pathtoPublish: '$(agent.buildDirectory)/output/$(sdk)/$(configuration)/s.ipa'
artifactName: 'ipaDrop'
Bash file
#!/usr/bin/env bash
# filename: pipeline-entitlements.sh
echo "Editing build.xcconfig"
TARGET_KEY="CODE_SIGN_ENTITLEMENTS"
REPLACEMENT_VALUE="s\/Resources\/YOURENTITLEMENTFILENAME.entitlements"
CONFIG_FILE="./app/App_Resources/iOS/build.xcconfig"
echo "Editing $TARGET_KEY and replaceing value with $REPLACEMENT_VALUE"
sed -i.bak "s/\($TARGET_KEY *= *\).*/\1$REPLACEMENT_VALUE/" $CONFIG_FILE
echo "Finished editing build.xcconfig"
I am trying to build/deploy an angular app everytime I push my code to github. The travis CI build passes but it doesn't deploy to Surge for some reason. I added SURGE_LOGIN and SURGE_TOKEN environment variables to the repository settings but it still doesn't work. Should it not mention that deployment failed in the build log? Any idea what is going wrong here/how I can fix this? Also I don't get emails when a build passes/fails even though this is set in the config file and my email address i added to the SURGE_LOGIN environment variable
the end of my travis build log:
42.29s$ ng build -prod
1439
Date: 2017-11-02T11:04:25.130Z
1440
Hash: 5dadab3e49327d48aac1
1441
Time: 38060ms
1442
chunk {0} polyfills.d8d3d78a4deb2ab66856.bundle.js (polyfills) 66.1 kB {4} [initial] [rendered]
1443
chunk {1} styles.4d93494871bdc47b353f.bundle.css (styles) 115 kB {4} [initial] [rendered]
1444
chunk {2} main.725afabe80d80f10fd14.bundle.js (main) 8.08 kB {3} [initial] [rendered]
1445
chunk {3} vendor.4400ceca3ce00f041a26.bundle.js (vendor) 434 kB [initial] [rendered]
1446
chunk {4} inline.fe3295955bbd9314430c.bundle.js (inline) 1.45 kB [entry] [rendered]
1447
1448
1449
The command "ng build -prod" exited with 0.
1450
1451
Done. Your build exited with 0.
my .travis.yml code:
#travis CI build configuration
#build language
language: node_js
#node_js versions
node_js:
- "6.11.2"
#before running the build
before-script:
- npm install #install all dependencies
- npm install -g surge #global surge install
#actual build step
script:
- ng build -prod
#build only on push not on pull requests.
deploy:
provider: surge
skip_cleanup: true
project: ./dist/ #build output path
domain: gaping-feeling.surge.sh #surge domain
#notifications
notifications:
email:
on_success: change #default: change
on_failure: change #default: change
Your issue is the yaml indentation. Yaml is indentation specific. So
a:
b:
and
a:
b:
They both have different meaning. In first a and b are top level attributes which in later case b is a child attribute of a. Your yaml should be
#travis CI build configuration
#build language
language: node_js
#node_js versions
node_js:
- "6.11.2"
#before running the build
before-script:
- npm install #install all dependencies
- npm install -g surge #global surge install
#actual build step
script:
- ng build -prod
#build only on push not on pull requests.
deploy:
provider: surge
skip_cleanup: true
project: ./dist/ #build output path
domain: gaping-feeling.surge.sh #surge domain
notifications:
email:
on_success: change
on_failure: change
If you still face issues let me know. But this should fix the problem
I installed Docker on my Synology NAS (DS415+) and tried to run the handbrake-cli (via this package) over ssh.
However, something seems to be broken. I get the following error message after a simple sudo docker run -d supercoder/docker-handbrake-cli -i ~/_inProgress/input/movie.mkv -o ~/_inProgress/output/test.mp4 (I shortened the error message for readability):
- hb_init: starting libhb thread
- HandBrake 0.10.1 (2015030800) - Linux x86_64 - https://handbrake.fr
- 4 CPUs detected
- Opening /var/services/homes/xxx/_inProgress/input/movie.mkv...
- CPU: Intel(R) Atom(TM) CPU C2538 # 2.40GHz
- Intel microarchitecture Silvermont
- logical processor count: 4
- OpenCL: library not available
- hb_scan: path=/var/services/homes/xxx/_inProgress/input/movie.mkv, title_index=1
- libbluray/bdnav/index_parse.c:162: indx_parse(): error opening /var/services/homes/xxx/_inProgress/input/movie.mkv/BDMV/index.bdmv
- libbluray/bdnav/index_parse.c:162: indx_parse(): error opening /var/services/homes/xxx/_inProgress/input/movie.mkv/BDMV/BACKUP/index.bdmv
- libbluray/bluray.c:2182: nav_get_title_list(/var/services/homes/xxx/_inProgress/input/movie.mkv) failed
- bd: not a bd - trying as a stream/file instead
- libdvdnav: Using dvdnav version 5.0.1
- libdvdread: Encrypted DVD support unavailable.
- libdvdread: Can't stat /var/services/homes/xxx/_inProgress/input/movie.mkv
- No such file or directory
- libdvdread: Could not open /var/services/homes/xxx/_inProgress/input/movie.mkv
- libdvdnav: vm: failed to open/read the DVD
- dvd: not a dvd - trying as a stream/file instead
- hb_stream_open: open /var/services/homes/xxx/_inProgress/input/movie.mkv failed
- scan: unrecognized file type
- libhb: scan thread found 0 valid title(s)
- No title found.
- HandBrake has exited.
I followed this blog post originally and got the same message there.
Executing the same thing on my desktop works without any problems.
Anyone got an idea?
When running the docker instance, your input and output file do not exist in the container. You need first to mount the input and output directories of you file system as volumes (as shown in the blog post you shared)
-v ~/_inProgress/output/:/outout:rw
-v ~/_inProgress/input/:/input:ro
And then you use those paths in the options:
-i /input/<file>
-o /output/<file>
Good luck!
I am experimenting with creating an EC2 instance to host a Perforce server. My instance is configured with the following user data:
#!/bin/bash
# Add a newline to the ec2-user prompt string
echo PS1=\"\\n\$PS1\" >> /home/ec2-user/.bashrc
# Update all packages
yum update –y
# Install Perforce packages
# The RHEL/7 part of the baseurl should be replaced with
# the latest RHEL version that both Amazon and Perforce support
rpm –import https://package.perforce.com/perforce.pubkey
cd /etc/yum.repos.d/
echo [perforce] > perforce.repo
echo name=Perforce >> perforce.repo
echo baseurl=http://package.perforce.com/yum/rhel/7/x86_64 >> perforce.repo
echo enabled=1 >> perforce.repo
echo gpgcheck=1 >> perforce.repo
yum install –y helix-p4d
# Make directories for the server, owned by new “perforce” user
cd /opt/perforce/servers/
mkdir danware
cd danware
mkdir danware-db danware-chkpts journal
chown –R perforce:perforce danware
I have tested each of the above commands, and know that they work when executed manually in this order. However, some aspect of Amazon's base64 encode/decode system seems to be getting in the way. When I go to "Actions > Instance Settings > View/Change User Data" from the EC2 Console after launching (and passing all system checks), I see the following user data. Note how almost every hyphen "-" has been replaced with some strange "a" character.
However, I'm not sure that this is the issue, because the log file at /var/log/cloud-init-output.log gives me the following output (I replaced some repetitive text with [...] to save space). Note the line that says Failed running /var/lib/cloud/instance/scripts/part-001 I have verified that this part-001 file actually does have the correctly displayed hyphen characters.
[...]
Cloud-init v. 0.7.6 running 'modules:final' at Fri, 09 Sep 2016 06:23:39 +0000. Up 86.66 seconds.
Loaded plugins: priorities, update-motd, upgrade-helper
No Match for argument: –y
No packages marked for update
RPM version 4.11.2
Copyright (C) 1998-2002 - Red Hat, Inc.
This program may be freely redistributed under the terms of the GNU GPL
Usage: rpm [-aKfgpqVcdLilsiv?] [-a|--all] [-f|--file] [-g|--group] [...]
Loaded plugins: priorities, update-motd, upgrade-helper
Resolving Dependencies
--> Running transaction check
---> [...]
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
helix-p4d x86_64 2016.1-1429894 perforce 24 k
Installing for dependencies:
helix-cli x86_64 2016.1-1429894 perforce 8.8 k
helix-cli-base x86_64 2016.1-1429894 perforce 1.4 M
helix-p4d-base x86_64 2016.1-1429894 perforce 3.1 k
helix-p4d-base-16.1 x86_64 2016.1-1429894 perforce 2.4 M
helix-p4dctl x86_64 2016.1-1429894 perforce 1.2 M
Transaction Summary
================================================================================
Install 1 Package (+5 Dependent packages)
Total download size: 5.0 M
Installed size: 13 M
Is this ok [y/d/N]: Exiting on user command
Your transaction was saved, rerun it with:
yum load-transaction /tmp/yum_save_tx.2016-09-09.06-23.dRP_r2.yumtx
/var/lib/cloud/instance/scripts/part-001: line 22: cd: /opt/perforce/servers/: No such file or directory
chown: invalid user: ‘–R’
Sep 09 06:23:41 cloud-init[2517]: util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-001 [1]
Sep 09 06:23:41 cloud-init[2517]: cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
Sep 09 06:23:41 cloud-init[2517]: util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python2.7/dist-packages/cloudinit/config/cc_scripts_user.pyc'>) failed
Cloud-init v. 0.7.6 finished at Fri, 09 Sep 2016 06:23:41 +0000. Datasource DataSourceEc2. Up 88.53 seconds
Even more annoying, I assumed that the early No Match for argument: –y line from the log file was referring to the yum update -y line from my user data. Sure enough, just running the example user data script from the EC2 documentation page, which also uses yum update -y, gives me this same error/warning! Amazon's own example script doesn't work!? So can anyone answer why A) AWS is not displaying the user data code correctly, and B) why my user data is yielding the errors shown above? The help is much appreciated!
For lines such as
yum update –y
The character you are using is a "EN DASH U+2013"
The usual character for a hyphen is "HYPHEN-MINUS U+002D"
Fix your user data source to use "hyphen minus" and have another go
I checked the character codes by cut n pasting into this online site http://www.fileformat.info/info/unicode/char/search.htm?q=-&preview=entity
Don't know if you can see the difference but this is your hyphen
yum update –y
and this is a "hyphen minus"
yum update -y