I am trying to access some s3 buckets from an instance created applying the IAM profile that allows full access to those s3 buckets. I am able to carry out the required operations using aws cli. However, my application is written in Ruby and makes use of fog gem. Using fog, I am unable to access those buckets. All I get is Access Denied. Additionally, I grabbed the required keys:
aws_access_key_id=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | grep AccessKeyId | cut -d':' -f2 | sed 's/[^0-9A-Z]*//g'`
aws_secret_access_key=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | grep SecretAccessKey | cut -d':' -f2 | sed 's/[^0-9A-Za-z/+=]*//g'`
and tried to access the buckets. Again, access denied.
Is there anything I should have missed?
As well as an access key and a secret access key, temporary credentials such as the ones provided by instance metadata also have a session token - without the token the credentials are invalid.
Current versions of fog / fog-aws support fetching instance credentials for you, eg
storage = Fog::Storage::AWS.new(region: "eu-west-1", use_iam_profile:true)
This will also refetch credentials before they expire
Related
I need to get the public IPs of all instances of my OCI tenant.
Here i saw a python scripts to do this from OCI Console Cloud Shell : https://medium.com/oracledevs/writing-python-scripts-to-run-from-the-oci-console-cloud-shell-a0be1091384c
But I want to create a bash script, that uses OCI CLI commands to fetch the required data.
How can I achieve this using OCI CLI commands?
OCI CLI structured-search and query feature can be used to fetch the OCID of instances, and instance command can be used fetch the instance details.
The output would be in json format by default.
You can use jq to filter needed data from the output json and create an array with it.
(OCI tool supports JMESPath queries)
Here is the snippet from bash script that uses OCI CLI commands to get public IPs of all compute instances in the compartment :
Pre-requisites: OCI CLI should be installed and configured properly to authenticate with the correct tenant and compartment
# Fetch the OCID of all the running instances in OCI and store to an array
instance_ocids=$(oci search resource structured-search --query-text "QUERY instance resources where lifeCycleState='RUNNING'" --query 'data.items[*].identifier' --raw-output | jq -r '.[]' )
# Iterate through the array to fetch details of each instance one by one
for val in ${instance_ocids[#]} ; do
echo $val
# Get name of the instance
instance_name=$(oci compute instance get --instance-id $val --raw-output --query 'data."display-name"')
echo $instance_name
# Get Public Ip of the instance
public_ip=$(oci compute instance list-vnics --instance-id $val --raw-output --query 'data[0]."public-ip"')
echo $public_ip
done
References :
https://blogs.oracle.com/cloud-infrastructure/post/exploring-the-search-and-query-features-of-oracle-cloud-infrastructure-command-line-interface
https://docs.oracle.com/en-us/iaas/tools/oci-cli/2.9.9/oci_cli_docs/cmdref/compute/instance.html
The developer who created a platform my company uses is no longer working for us and I don't know how I can retrieve the passwords from a custom PHP application
When I look in the PHPmyAdmin the passwords are ecrypted (eg *2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19)
How can I change or retrieve these?
If a proper encryption method was used, it's not going to be possible to easily retrieve them.
Just reset them with new passwords.
Edit: The string looks like it is using PASSWORD():
UPDATE user SET password = PASSWORD("newpassword");
How can I decrypt MySQL passwords
You can't really because they are hashed and not encrypted.
Here's the essence of the PASSWORD function that current MySQL uses. You can execute it from the sql terminal:
mysql> SELECT SHA1(UNHEX(SHA1("password")));
+------------------------------------------+
| SHA1(UNHEX(SHA1("password"))) |
+------------------------------------------+
| 2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19 |
+------------------------------------------+
1 row in set (0.00 sec)
How can I change or retrieve these?
If you are having trouble logging in on a debian or ubuntu system, first try this (thanks to tohuwawohu at https://askubuntu.com/questions/120718/cant-log-to-mysql):
$ sudo cat /etc/mysql/debian.conf | grep -i password
...
password: QWERTY12345...
Then, log in with the debian maintenance user:
$ mysql -u debian-sys-maint -p
password:
Finally, change the user's password:
mysql> UPDATE mysql.user SET Password=PASSWORD('new password') WHERE User='root';
mysql> FLUSH PRIVILEGES;
mysql> quit;
When I look in the PHPmyAdmin the passwords are encrypted
Related, if you need to dump the user database for the relevant information, try:
mysql> SELECT User,Host,Password FROM mysql.user;
+------------------+-----------+----------------------+
| User | Host | Password |
+------------------+-----------+----------------------+
| root | localhost | *0123456789ABCDEF... |
| root | 127.0.0.1 | *0123456789ABCDEF... |
| root | ::1 | *0123456789ABCDEF... |
| debian-sys-maint | localhost | *ABCDEF0123456789... |
+------------------+-----------+----------------------+
And yes, those passwords are NOT salted. So an attacker can prebuild the tables and apply them to all MySQL installations. In addition, the adversary can learn which users have the same passwords.
Needles to say, the folks at mySQL are not following best practices. John Steven did an excellent paper on Password Storage Best Practice at OWASP's Password Storage Cheat Sheet. In fairness to the MySQL folks, they may be doing it because of pain points in the architecture, design or implementation (I simply don't know).
If you use the PASSWORD and UPDATE commands and the change does not work, then see http://dev.mysql.com/doc/refman/5.0/en/resetting-permissions.html. Even though the page is named "resetting permissions", its really about how to change a password. (Its befuddling the MySQL password change procedure is so broken that you have to jump through the hoops, but it is what it is).
Hashing is a one-way process but using a password-list you can regenerate the hashes and compare to the stored hash to 'crack' the password.
This site https://crackstation.net/ attempts to do this for you - run through passwords lists and tell you the cleartext password based on your hash.
With luck, if the original developer was any good, you will not be able to get the plain text out. I say "luck" otherwise you probably have an insecure system.
For the admin passwords, as you have the code, you should be able to create hashed passwords from a known plain text such that you can take control of the application. Follow the algorithm used by the original developer.
If they were not salted and hashed, then make sure you do apply this as 'best practice'
just change them to password('yourpassword')
You can't decrypt MySQL passwords, because the are hashed by using MD5 hash algorithm, which is not an encryption algorithm.
Simply best way from linux server
sudo mysql --defaults-file=/etc/mysql/debian.cnf -e 'use mysql;UPDATE user SET password=PASSWORD("snippetbucket-technologies") WHERE user="root";FLUSH PRIVILEGES;'
This way work for any linux server, I had 100% sure on Debian and Ubuntu you win.
I have 2 instances of the same deployment/project on AWS Elastic Beanstalk.
Both contain a Laravel project which contains scheduling code which runs various commands which can be found in the schedule method/function of the Kernel.php class within 'app/Console' - the problem I have is that if a command runs from one instance then it will also run the command from the second instance which is not what I want to happen.
What I would like to happen is that the commands get run from only one instance and not the other. How do I achieve this in the easiest way possible?
Is there a Laravel package which could help me achieve this?
From Laravel 5.6:
Laravel provides a onOneServer method which you can use if your applications share a single cache server. You could use something like ElastiCache to host Redis or Memcached and use it as your cache server for both of your application instances. Then you would be able to use the onOneServer method like this:
$schedule->command('report:generate')
->fridays()
->at('17:00')
->onOneServer();
For older versions of Laravel:
You could use the jdavidbakr/multi-server-event package. Once you have it set up you should be able to use it like:
$schedule->command('inspire')
->daily()
->withoutOverlappingMultiServer();
I had the same issue to run some cronjobs (nothing related to Laravel) and I found a nice solution (don't remember where I found it)
What I do is check if the instance running the code is the first instance on the Auto Scaling Group, if it's the first then I execute the command otherwise just exit.
This is the way it's implemented:
#!/bin/bash
INSTANCE_ID=`curl http://169.254.169.254/latest/meta-data/instance-id 2>/dev/null`
REGION=`curl -s http://169.254.169.254/latest/dynamic/instance-identity/document 2>/dev/null | jq -r .region`
# Find the Auto Scaling Group name from the Elastic Beanstalk environment
ASG=`aws ec2 describe-tags --filters "Name=resource-id,Values=$INSTANCE_ID" \
--region $REGION --output json | jq -r '.[][] | select(.Key=="aws:autoscaling:groupName") | .Value'`
# Find the first instance in the Auto Scaling Group
FIRST=`aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names $ASG \
--region $REGION --output json | \
jq -r '.AutoScalingGroups[].Instances[] | select(.LifecycleState=="InService") | .InstanceId' | sort | head -1`
# If the instance ids are the same exit 0
[ "$FIRST" = "$INSTANCE_ID" ]
Try implementing those calls using PHP and it should work.
When creating a IAM user for salt-cloud, what are the minimum rights to give it so that it can do its job following the principle of least privilege?
I just need to create EC2 instances, using a map file: however I don't know enough salt-cloud to be sure of the actual operations it performs.
I would prefer to use a predefined policy, if it exists.
I was curious about this myself so took a look in the salt-cloud source (salt/cloud/clouds/ec2.py). Interestingly they don't use the boto library to make calls to AWS, opting instead to carve the requests themselves and the way they've done it makes it really easy to extract the actions that you'd need to have permissions for.
This one-liner pulls out all the actions
grep "'Action':" cloud/clouds/ec2.py | awk '{print $4;}' | sed "s/[},']//g" | sort | uniq
As you can see it's a fairly substantial subset of all the available EC2 permissions.
AllocateAddress
AssociateAddress
AttachVolume
CancelSpotInstanceRequests
CopySnapshot
CreateKeyPair
CreateSnapshot
CreateTags
CreateVolume
DeleteKeyPair
DeleteSnapshot
DeleteTags
DeleteVolume
DescribeAvailabilityZones
DescribeImages
DescribeInstanceAttribute
DescribeInstanceTypes
DescribeInstances
DescribeKeyPairs
DescribeRegions
DescribeSnapshots
DescribeSpotInstanceRequests
DescribeSubnets
DescribeTags
DescribeVolumes
DescribeZones
DetachVolume
GetConsoleOutput
GetPasswordData
ImportKeyPair
ModifyInstanceAttribute
ModifyNetworkInterfaceAttribute
RebootInstance
RebootInstances
RegisterImage
RequestSpotInstances
RunInstances
StartInstance
StartInstances
StopInstances
TerminateInstances
Naturally you could use salt to create an IAM profile for you with these ;)
I'm attempting to use the new incremental authorization for an installed app in order to add scopes to an existing authorization while keeping the existing scopes. This is done using the new include_granted_scopes=true parameter. However, no matter what I've tried, the re-authorization always overwrites the scopes completely. Here's a minimal Bash PoC script I've written to demo my issue:
client_id='716905662885.apps.googleusercontent.com' # throw away client_id (non-prod)
client_secret='CMVqIy_iQqBEMlzjYffdYM8A' # not really a secret
redirect_uri='urn:ietf:wg:oauth:2.0:oob'
while :
do
echo "Please enter a list of scopes (space separated) or CTRL+C to quit:"
read scope
# Form the request URL
# http://goo.gl/U0uKEb
auth_url="https://accounts.google.com/o/oauth2/auth?scope=$scope&redirect_uri=$redirect_uri&response_type=code&client_id=$client_id&approval_prompt=force&include_granted_scopes=true"
echo "Please go to:"
echo
echo "$auth_url"
echo
echo "after accepting, enter the code you are given:"
read auth_code
# swap authorization code for access token
# http://goo.gl/Mu9E5J
auth_result=$(curl -s https://accounts.google.com/o/oauth2/token \
-H "Content-Type: application/x-www-form-urlencoded" \
-d code=$auth_code \
-d client_id=$client_id \
-d client_secret=$client_secret \
-d redirect_uri=$redirect_uri \
-d grant_type=authorization_code)
access_token=$(echo -e "$auth_result" | \
grep -Po '"access_token" *: *.*?[^\\]",' | \
awk -F'"' '{ print $4 }')
echo
echo "Got an access token of:"
echo $access_token
echo
# Show information about our access token
info_result=$(curl -s --get https://www.googleapis.com/oauth2/v2/tokeninfo \
-H "Content-Type: application/json" \
-d access_token=$access_token)
current_scopes=$(echo -e "$info_result" | \
grep -Po '"scope" *: *.*?[^\\]",' | \
awk -F'"' '{ print $4 }')
echo "Our access token now allows the following scopes:"
echo $current_scopes | tr " " "\n"
echo
echo "Let's add some more!"
echo
done
The script simply performs OAuth authorization and then prints out the scopes the token is currently authorized to use. In theory it should continue to add scopes each time through but in practice, the list of scopes is getting overwritten each time. So the idea would be on the first run, you'd use a minimal scope of something like email and then the next run, tack on something more like read-only calendar https://www.googleapis.com/auth/calendar.readonly. Each time, the user should only be prompted to authorize the currently requested scopes but the resulting token should be good for all scopes including those authorized on previous runs.
I've tried with a fresh client_id/secret and the results are the same. I know I could just include the already authorized scopes again but that prompts the user for all of the scopes, even those already granted and we all know the longer the list of scopes, the less likely the user is to accept.
UPDATE: during further testing, I noticed that the permissions for my app do show the combined scopes of each incremental authorization. I tried waiting 30 seconds or so after the incremental auth, then grabbing a new access token with the refresh token but that access token is still limited to the scopes of the last authorization, not the combined scope list.
UPDATE 2: I've also toyed around with keeping the original refresh token. The refresh token is only getting new access tokens that allow the original scopes, the incrementally added scopes are not included. So it seems effectively that include_granted_scopes=true is having no effect on the tokens, the old and new refresh tokens continue to work but only for their specified scopes. I cannot get a "combined scope" refresh or access token.
Google's OAuth 2.0 service does not support incremental auth for installed/native apps; it only works for the web server case. Their documentation is broken.
Try adding a complete list of scopes to the second request, where you exchange authorization code for an access token. Strangely enough, scope parameter doesn't seem to be documented, but it is present in requests generated by google-api-java-client. For example:
code=foo&grant_type=authorization_code
&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Fmyapp%2FoauthCallback
&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fplus.me+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fplus.stream.write
In the web server scenario, a complete list of granted scopes is returned together with authorization code when include_granted_scopes is set to true. This is another bit of information that seems to be missing from linked documentation.
Edit 1 Including a complete list of scopes in the code exchange request works for us in our Java app, but I have just tried your original script with no modification (except for client id/secret) and it works just fine (edited just the ids and tokens):
$ bash tokens.sh
Please enter a list of scopes (space separated) or CTRL+C to quit:
https://www.googleapis.com/auth/userinfo.profile
Please go to:
https://accounts.google.com/o/oauth2/auth?scope=https://www.googleapis.com/auth/userinfo.profile&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&client_id=189044568151-4bs2mcotfi2i3k6qp7vq8c6kbmkp2rf8.apps.googleusercontent.com&approval_prompt=force&include_granted_scopes=true
after accepting, enter the code you are given:
4/4qXGQ6Pt5QNYqdEuOudzY5G0ogru.kv_pt5Hlwq8UYKs_1NgQtlUFsAJ_iQI
Got an access token of:
ya29.1.AADtN_XIt8uUZ_zGZEZk7l9KuNQl9omr2FRXYAqf67QF92KqfvXliYQ54ffg_3E
Our access token now allows the following scopes:
https://www.googleapis.com/auth/userinfo.profile
https://www.googleapis.com/auth/userinfo.email
https://www.googleapis.com/auth/plus.me
https://www.googleapis.com/auth/plus.circles.read
You can see that the previously granted scopes are included...