AWS CodeDeploy script does not exist during AfterInstall hook - amazon-ec2

I am having trouble with executing my install_dependencies.sh script in AfterInstall hook-
it seems like my deployment-archive from which CodeDeploy agent is trying to run the scripts from does not yet exists during the AfterInstall hook.
I am getting this error that causes my deployment to fail:
ERROR [codedeploy-agent(2643)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Error during perform:
InstanceAgent::Plugins::CodeDeployPlugin::ScriptError -
Script does not exist at specified location: /opt/codedeploy-agent/deployment-root/432f58a2-27d7-46c0-8091-1625eb8733af/d-Q3DGM6LPU/deployment-archive/scripts/install_dependencies.sh -
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/hook_executor.rb:118:in `block (2 levels) in execute'...
my appspec.yml file looks like this:
version: 0.0
os: linux
files:
- source: /
destination: /home/ec2-user/myApp
hooks:
AfterInstall:
- location: scripts/install_dependencies.sh
timeout: 300
runas: root
ApplicationStop:
- location: scripts/stop_server.sh
timeout: 300
runas: root
ApplicationStart:
- location: scripts/start_server.sh
timeout: 300
runas: root
for accessing the deployment-archive folder after the failed deployment attempt, i do get files in their correct path (the same path that appeared in the Error log):
[ec2-user#ip-172-31-10-87 scripts]$ cd /opt/codedeploy-agent/deployment-root/432f58a2-27d7-46c0-8091-1625eb8733af/d-Q3DGM6LPU/deployment-archive/scripts/
[ec2-user#ip-172-31-10-87 scripts]$ ls -ln
total 12
-rwxrwxrwx 1 0 0 261 Aug 21 13:33 install_dependecies.sh
-rwxrwxrwx 1 0 0 69 Aug 21 13:33 start_server.sh
-rwxrwxrwx 1 0 0 56 Aug 21 13:33 stop_server.sh
my EC2 instance os is-
[ec2-user#ip-172-31-10-87 ~]$ cat /etc/issue
Amazon Linux AMI release 2018.03
[ec2-user#ip-172-31-10-87 ~]$ uname -a
Linux ip-172-31-10-87 4.14.62-65.117.amzn1.x86_64 #1 SMP Fri Aug 10 20:03:52
UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
I have already checked for execution permissions, and the content of my script files is empty (with only "exit 0" statement) in order to help sandbox the problem.
as far as i've seen in all examples- the scripts folder is expected to be in my deployed package and not outside of my deployment.
my appspec.yml is in the root folder of my project, and so is my scripts folder.
if I eliminate the AfterInstall/BeforeInstall hooks from my appspec- the problem does not occur.
What am i missing? why does these file are not accessible during the AfterInstall/BeforeInstall hooks?
please help,
thanks

Related

Error When Importing ROS Projects & Workspace in CLion

I wanted to give CLion a try for working with the ROS source code. I created a ROS Workspace and have the following folder structure:
joesan#joesan-InfinityBook-S-14-v5:~/Projects/Private/ros-projects$ cd catkin_ws/
drwxrwxr-x joesan joesan 4 KB Sun Aug 30 10:48:27 2020  .
drwxrwxr-x joesan joesan 4 KB Sun Aug 30 10:48:10 2020  ..
.rw-rw-r-- joesan joesan 98 B Sun Aug 30 10:48:27 2020  .catkin_workspace
drwxrwxr-x joesan joesan 4 KB Sun Aug 30 10:48:27 2020  build
drwxrwxr-x joesan joesan 4 KB Sun Aug 30 10:48:26 2020  devel
drwxrwxr-x joesan joesan 4 KB Sun Aug 30 12:13:50 2020  src
joesan#joesan-InfinityBook-S-14-v5:~/Projects/Private/ros-projects/catkin_ws$
I have sourced this location in the .bash_profile as below:
# Dynamically source all setup.bash files from multiple catkin ros workspaces
find /home/joesan/Projects/Private/ros-projects -wholename '*/devel/setup.bash' | xargs source
I then source the .bash_profile from within my .bashrc
Now, when I tried to open the project in CLion, I see the following error:
CMake Error at CMakeLists.txt:65 (message):
find_package(catkin) failed. catkin was neither found in the workspace nor
in the CMAKE_PREFIX_PATH. One reason may be that no ROS setup.sh was
sourced before.
I launch CLion via a Desktop shortcut entry that I created. How do I make CLion aware of this .bash_profile such that it can find catkin?
I managed to fix this by doing the following, but make sure that you have a Desktop entry created for Jetbrains CLion beforehand!
Open the jetbrains-clion.desktop located in the .local folder:
~/.local/share/applications/jetbrains-clion.desktop
In this desktop entry, modify the Exec= line as:
Exec=bash -i -c "/opt/softwares/clion-2020.2.1/bin/clion.sh" %f

After deleting Gradle's build cache and building, some tasks are still marked as FROM-CACHE

I start off by deleting Gradle's build cache
$ rm -rf $GRADLE_HOME/caches/build-cache-*
I manually confirm this directory is gone. I also clean to ensure nothing from incremental build will interfere
$ ./gradlew clean
I then run a build w/ build-cache enabled:
$ ./gradlew --build-cache --scan app:assembleDebug
...
238 actionable tasks: 172 executed, 49 from cache, 17 up-to-date
Checking the generated report I see that some tasks are still marked as FROM-CACHE. How is this happening?
FWIW I'm using Gradle 5.6.3. I also don't use a remote build cache.
UPDATE: after clearing build cache, here's what's left in the cache dir:
$ pwd
/Users/jasonatwood/.gradle/caches
$ ls -al
total 24
drwx------ 11 jasonatwood staff 352 Dec 13 18:11 .
drwxr-xr-x 9 jasonatwood staff 288 Oct 15 17:24 ..
drwxr-xr-x 14 jasonatwood staff 448 Oct 29 16:07 5.4.1
drwxr-xr-x 15 jasonatwood staff 480 Nov 26 18:25 5.6.3
drwxr-xr-x 35 jasonatwood staff 1120 Dec 11 14:14 jars-3
drwxr-xr-x 5 jasonatwood staff 160 Oct 15 17:44 journal-1
drwxr-xr-x 7 jasonatwood staff 224 Oct 15 17:50 modules-2
drwxr-xr-x 5 jasonatwood staff 160 Oct 15 17:50 transforms-2
-rw-r--r-- 1 jasonatwood staff 39 Oct 15 17:27 transforms-2.lock
-rw------- 1 jasonatwood staff 28 Oct 22 16:34 user-id.txt
-rw-r--r-- 1 jasonatwood staff 17 Dec 13 18:00 user-id.txt.lock
If I understand correctly, I don't need to delete any of these.
The local task cache in the .gradle directory within the root project might interfere.
In order to disable the local/global caching altogether, use gradle.properties:
org.gradle.caching=false
If you remove the local build cache and don't use a remove cache, Gradle should indeed not be able to fetch anything cached on the next build. However, I don't think you actually removed anything.
The GRADLE_HOME environment variable, if you have it, should points to an unpacked local Gradle distribution. It could be something like /Users/jasonatwood/gradle-6.0-all. It would be wrong to set it to a path that is not a Gradle distribution.
The Gradle build cache resides in /Users/jasonatwood/.gradle/caches/build-cache-1.
When you try to remove it with $ rm -rf $GRADLE_HOME/caches/build-cache-*, you are using GRADLE_HOME as if it is pointing to the root of your home folder (/Users/jasonatwood), which it most likely isn't.
So instead, use this for removing it: $ rm -rf ~/.gradle/caches/build-cache-*
Or, at the very least, verify that the cache folder has indeed been removed.
If I run a --scan and look at the FROM-CACHE tasks in the generated HTML report.
I then look at the build cache keys for those FROM-CACHE tasks. I then go find those keys on my local machine. I see they are generated at the same time as the rest of the build. So I'm guessing that Gradle is using build cache within a build and not just across builds.
Sometimes there is a need to invalidate cache every time you restart the IDEA. This may be due to shared indexes reuse. What I did Settings --> Tools --> Shared Indexes -> Download Automatically and it resolved all the pain.
Earlier the setting was Don't download use Local.

My metasploit module(s) not found/loading

Any custom metasploit module I create isn't getting loaded.
I tried both of these demos:
https://www.offensive-security.com/metasploit-unleashed/building-module/ https://github.com/rapid7/metasploit-framework/wiki/Loading-External-Modules
and got the same result that the modules were NOT found.
Before posting here, I checked these out:
https://forums.kali.org/showthread.php?28940-Metasploit-modules-not-loading!
https://www.offensive-security.com/metasploit-unleashed/modules-and-locations
and from SO:
I can't get new modules to load in metasploit
How to add module in Metasploit?
no help
Just working with the github example, on the Kali host, I do indeed have the file in the right location (according to the demo):
root#kali:~/.msf4/modules/exploits/test# ls -al
total 12
drwxr-xr-x 2 root root 4096 Mar 19 13:59 .
drwxr-xr-x 3 root root 4096 Mar 19 13:58 ..
-rw-r--r-- 1 root root 9 Mar 19 13:59 test_module.rb
I then ran reload_all and when using this command:
use exploit/test/test_module it returns with Failed to load module.
I also tried to manually load that path and it failed too:
msf > loadpath ~/.msf4/modules/
Loaded 0 modules:
Posting the answer for anyone who may come across this. I'm kinda new to this; didn't know where metasploit info gets logged.
It's in: ~/.msf4/logs/framework.log
The log told me I had a syntax error... must have happened from copying and pasting the text from the browser. Cleaned that up and everything works.

Maven error: Could not find or load main class .usr.share.maven.boot.plexus-classworlds-2.x.jar

I have recently upgraded my Ubuntu from 14.04 to 16.04 and started getting the following error when running any mvn commands (version 3.3.9): Error: Could not find or load main class .usr.share.maven.boot.plexus-classworlds-2.x.jar. My environment variables are declared as follows:
$JAVA_HOME: /usr/lib/jvm/java-8-oracle
$PATH: /usr/local/texlive/2015/bin/x86_64-linux:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
$M2_HOME: /usr/share/maven
$M2: /usr/share/maven/bin
When trying to find a solution, I've tried removing the M2 and M2_HOME variables as suggested on various threads, which resulted in getting a different error: Error: Could not find or load main class org.codehaus.plexus.classworlds.launcher.Launcher. I have also tried running apt-get remove --purge maven and installing it again as well as downloading the .tar.gz archive, but in both cases, nothing has changed.
When looking into the /usr/share/maven/boot folder, there is a chain of symlinks pointing from plexus-classworlds-2.x.jar -> /usr/share/java/plexus-classworlds2-2.5.2.jar. Am I missing some dependencies? Or are there any old configuration files that did not get removed by --purge?
EDIT: When I execute mvn as root, I get the Launcher error instead of plexus-classworlds-2.x. Also, I have completely removed and reinstalled all plexus libraries, yet with no change.
Check if /usr/share/maven/boot has multiple jars with pattern /usr/share/maven/boot/plexus-classworlds-*.jar.
Mine was something like:
drwxr-xr-x 2 root root 4096 Dec 23 14:21 ./
drwxr-xr-x 6 root root 4096 Nov 14 2015 ../
-rw-r--r-- 1 root root 52684 Dec 12 2015 plexus-classworlds-2.5.2.jar
lrwxrwxrwx 1 root root 34 Dec 10 2015 plexus-classworlds-2.x.jar -> ../../java/plexus-classworlds2.jar
This messes up /usr/share/maven/bin/mvn bash script and executes a wrong java command.
I had to remove the jar and leave only the symbolic link.

Phusion Passenger, mod_rails rails 2.3 problem

"The application has exited during startup (i.e. during the evaluation of config/environment.rb). The error message may have been written to the web server's log file. Please check the web server's log file (i.e. not the (Rails) application's log file) to find out why the application exited."
I'm getting the above error message when I try to access my test site.
Permission for config/* is
-rw-r--r-- 1 www-data root 2011 Aug 27 20:17 environment.rb
drwxr-xr-x 3 root root 4096 Aug 27 20:17 environments
drwxr-xr-x 3 root root 4096 Aug 27 20:17 initializers
drwxr-xr-x 3 root root 4096 Aug 27 20:17 locales
-rw-r--r-- 1 root root 2431 Aug 27 20:17 routes.rb
I've tried "root root 2011 Aug 27 20:17 environment.rb" as well but no success either.
I'm running Debian GNU/Linux and have other Rails sites running fine on the same server, I have tried to replicate the setup from one of the other sites, but it's not having any effect.
I encountered this same issue, and after killing passenger and apache ended up finding:
http://geminstallthat.wordpress.com/2009/03/01/passenger-with-rails-23-application-missingsourcefile/
Essentially the issue is for some versions of libapache2-mod-passenger the app/controllers/application.rb change to app/controllers/application_controller.rb (in Rails 2.3) explodes things (technical term).

Resources