When using gsutil to copy a file it hangs indefinitely:
gsutil cp gs://gcp-public-data-sentinel-2/index.csv.gz .
Running gsutil with with debugging gsutil -D cp outputs:
gsutil version: 4.28n
boto version: 2.48.0
python version: 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
OS: Linux 2.6.32-504.23.4.el6.x86_64
multiprocessing available: True
using cloud sdk: True
pass cloud sdk credentials to gsutil: True
config path(s): /home/gardnera/.config/gcloud/legacy_credentials/my#email/.boto
gsutil path: /u/devon0/gardnera/yes/google-cloud-sdk/platform/gsutil/gsutil
compiled crcmod: True
installed via package manager: False
editable install: False
Command being run: /u/devon0/gardnera/yes/google-cloud-sdk/platform/gsutil/gsutil -o GSUtil:default_project_id=landsatdata1 -D cp gs://gcp-public-data-sentinel-2/index.csv.gz /home/gardnera/
config_file_list: ['/home/gardnera/.config/gcloud/legacy_credentials/my#email.com/.boto']
config: [('debug', '0'), ('working_dir', '/mnt/pyami'), ('https_validate_certificates', 'true'), ('debug', '0'), ('working_dir', '/mnt/pyami'), ('default_project_id', 'landsatdata1')]
INFO 0110 17:16:21.952356 base_api.py] Calling method storage.objects.get with StorageObjectsGetRequest: <StorageObjectsGetRequest
bucket: u'gcp-public-data-sentinel-2'
object: u'index.csv.gz'
projection: ProjectionValueValuesEnum(noAcl, 1)>
INFO 0110 17:16:21.953642 base_api.py] Making http GET to https://www.googleapis.com/storage/v1/b/gcp-public-data-sentinel-2/o/index.csv.gz?fields=contentType%2Cname%2Cgeneration%2CcustomerEncryption%2Cmd5Hash%2CcontentEncoding%2Ccrc32c%2Cetag%2CmediaLink%2Csize&alt=json&projection=noAcl
INFO 0110 17:16:21.954131 base_api.py] Headers: {'accept': 'application/json',
'accept-encoding': 'gzip, deflate',
'content-length': '0',
'user-agent': 'apitools gsutil/4.28 Python/2.7.13 (linux2) google-cloud-sdk/184.0.0 analytics/disabled'}
INFO 0110 17:16:21.954305 base_api.py] Body: (none)
INFO 0110 17:16:21.954441 client.py] Attempting refresh to obtain initial access_token
but hangs indefinitely on obtain initial access_token
I've been digging into this issue for awhile now and can't seem to find any answer. Testing on other machines I am unable to replicate the issue. A reinstall of gcloud did not fix the issue.
Anyone have any thoughts as to what might be going on?
Thanks
OK looks like the problem was with the .config files. I deleted the ~/.config/gcloud folder and the issue has resolved itself
Related
Getting into Minio. Investigating a few commands,
If I do mc ls alias/bucket then I get expected output:
[2020-12-09 19:48:15 UTC] 10B Account-9.dta
[2020-12-09 19:48:22 UTC] 10B Account-90.dta
[2020-12-09 19:48:22 UTC] 11B Account-92.dta
So, I would expect some kind of output when I execute the following on the same connection:
mc sql --recursive --query "select * from s3object" alias/bucket
but instead, it just goes back to a prompt (No results). I suspect my "from" is wrong but I have no idea what values to use other than "s3object".
How do I properly perform SQL queries on a local MinIO instance?
MinIO Version:
VERSION
2019-08-14T20:37:41Z
MEMORY
Used: 4.4 MB | Allocated: 3.6 GB | Used-Heap: 4.4 MB | Allocated-Heap: 65 MB
PLATFORM
Host: minio-66c9cd74c9-7m6lx | OS: linux | Arch: amd64
RUNTIME
Version: go1.12.8 | CPUs: 12
MinIO Client Version:
mc version RELEASE.2020-11-25T23-04-07Z
When you specify multiple objects, mc filters objects by extension.
If you store files with a .csv extension they should be picked up. Similar for .json files and gzip/bzip2.
I am trying to get the src and version details using the ansible-galaxy cli without using the internet for example :
[ansible#server exam]$ ansible-galaxy info bennojoy.ntp
Role: bennojoy.ntp
description: ansible role ntp
active: True
commit:
commit_message:
commit_url:
company: AnsibleWorks
created: 2013-12-19T01:25:11.835644Z
dependencies: []
download_count: 110946
forks_count: 0
galaxy_info:
author: Benno Joy
categories: ['development']
company: AnsibleWorks
license: BSD
min_ansible_version: 1.4
platforms: [{'name': 'EL', 'versions': [5, 6]}, {'name': 'Fedora', 'v>
github_branch: master
github_repo: ntp
github_user: bennojoy
id: 4
imported: None
install_date: Sun Jul 26 10:25:09 2020
installed_version:
is_valid: True
issue_tracker_url: https://github.com/bennojoy/ntp/issues
license: BSD
min_ansible_version: 1.4
modified: 2018-06-30T00:28:28.738481Z
open_issues_count: 0
path: ('/home/ansible/.ansible/roles', '/usr/share/ansible/roles', '/etc/ansi>
role_type: ANS
stargazers_count: 24
travis_status_url:
I am not able to get the src and version from the above. I am only able to get the src details upon installing as shown below :
[ansible#server exam]$ ansible-galaxy install bennojoy.ntp
downloading role from https://github.com/bennojoy/ntp/archive/master.tar.gz
extracting bennojoy.ntp to /home/ansible/.ansible/roles/bennojoy.ntp
bennojoy.ntp was installed successfully
In this case the src is https://github.com/bennojoy/ntp/archive/master.tar.gz. The version available for a role i need to check through the website. I would like to check if i can do this only via cli only by issuing some command instead ?
I have trouble with sphinxcontrib-httpdomain. If i defining http get or post or any another domain, sphinx-build tells me about error
Unknown directive type "http:get"
Definition of http:post domain in my sources:
.. http:get:: /users/(int:user_id)/posts/(tag)
The posts tagged with `tag` that the user (`user_id`) wrote.
**Example request**:
.. sourcecode:: http
GET /users/123/posts/web HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
[
{
"post_id": 12345,
"author_id": 123,
"tags": ["server", "web"],
"subject": "I tried Nginx"
},
{
"post_id": 12346,
"author_id": 123,
"tags": ["html5", "standards", "web"],
"subject": "We go to HTML 5"
}
]
:query sort: one of ``hit``, ``created-at``
:query offset: offset number. default is 0
:query limit: limit number. default is 30
:reqheader Accept: the response content type depends on
:mailheader:`Accept` header
:reqheader Authorization: optional OAuth token to authenticate
:resheader Content-Type: this depends on :mailheader:`Accept`
header of request
:statuscode 200: no error
:statuscode 404: there's no user
There are two blank lines before and after this block. I tried to use 3,4 spaces and to offset contents of block
Contents of extensions block of conf.py:
extensions = ['sphinx.ext.autodoc',
'sphinx.ext.extlinks',
'sphinx.ext.intersphinx',
'sphinx.ext.todo',
'sphinx.ext.coverage',
'sphinx.ext.ifconfig',
'sphinx.ext.doctest',
'sphinx.ext.graphviz',
'sphinxcontrib.httpdomain'
]
What i'd tried also and it doesn't helped:
- To reinstall all sphinx* packages via pip
To reinstall all sphinx* packages via apt-get
To specify path to extension in conf.py
To reinstall all autodoc, docutils packages via pip
To build on another host (same error)
To look at build logs with -vvv mode
To read [official plugin docs][3] and do all of steps
Some information which can be useful:
All packages are installed to /usr/local/lib/python2.7/dist-packages
pip show sphinx output:
Name: Sphinx
Version: 1.6.7
Summary: Python documentation generator
Home-page: http://sphinx-doc.org/
Author: Georg Brandl
Author-email: georg#python.org
License: BSD
Location: /usr/local/lib/python2.7/dist-packages
Requires: sphinxcontrib-websupport, babel, imagesize, setuptools, typing, Pygments, requests, six, docutils, alabaster, Jinja2, snowballstemmer
pip show sphinxcontrib-httpdomain output:
Name: sphinxcontrib-httpdomain
Version: 1.6.0
Summary: Sphinx domain for documenting HTTP APIs
Home-page: https://github.com/sphinx-contrib/httpdomain
Author: Hong Minhee
Author-email: hong.minhee#gmail.com
License: BSD
Location: /usr/local/lib/python2.7/dist-packages
Requires: six, Sphinx
uname output:
Linux 4.13.0-32-generic #35-Ubuntu SMP Thu Jan 25 09:13:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
After fighting with this problem for a while, I finally found the solution. The version included in Debian buster and Ubuntu bionic (and newer) has a patch added which solved an issue, but added this problem.
The patch comes from this commit, but it is missing the fix added a couple of days after in this other commit.
It seems that even if you tried to install with pip, you were using the system version. You can patch it trivially by editing the code at /usr/lib/python2/dist-packages/sphinxcontrib/httpdomain.py.
I am sending the patch to the package maintainer so it hopefully will be fixed soon in unstable.
Trying to run java_test that runs docker inside ProcessBuilder.
To simplify the code of the test is as following:
#Test
public void testDockerExecutable(){
System.out.println("======== running docker ==============");
try {
Process p = new ProcessBuilder("docker","version")
.inheritIO()
.start();
} catch (IOException e) {
e.printStackTrace();
throw new RuntimeException(e);
}
}
Running docker version straight from shell gives that output:
Client:
Version: 17.03.1-ce
API version: 1.27
Go version: go1.7.5
Git commit: c6d412e
Built: Tue Mar 28 00:40:02 2017
OS/Arch: darwin/amd64
Server:
Version: 17.03.1-ce
API version: 1.27 (minimum version 1.12)
Go version: go1.7.5
Git commit: c6d412e
Built: Fri Mar 24 00:00:50 2017
OS/Arch: linux/amd64
Experimental: true
But running the tests gives that output:
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time.
INFO: Found 1 test target...
JUnit4 Test Runner
.======== here ==============
java.io.IOException: Cannot run program "docker": error=2, No such file or directory
I know that I need to somehow import docker to the runfiles environment (just like local_jdk does). But how do I do that? Also - unlike jdk that only requires read permission, docker needs write permissions to it's lib folder.
My env is mac os x sierra and bazel HEAD (68028317c1d3d831a24f90e2b25d1410ce045c54).
tried it with java_test. The "local" attribute did not affect the failure. (tried it with both True and False).
update: works on Linux
I tried running this in linux and it works well both with "local"=True and "local"=False. Seems like it's something related to mac.
Linux sandbox mounts some directories by default (citing the docs):
We currently also mount /bin, /etc, /usr (except /usr/local), and every directory starting with /lib, to allow running local tools. In the future, we are planning to provide a shell with a set of Linux utilities, and to require that all other tools are specified as inputs.
I assume the docker binary is in one of the standard locations, and bazel finds it here.
Maybe on mac the binary is somewhere else and only exporting PATH to the test environment reveals it?
All aside, the best practice would be to make the test explicitly depend on some 'docker' target. Then bazel will make sure the binary is there. You can use local_repository (or its new_ variant) rules to hook it up.
Seems like this does the trick:
$ bazel test --test_env=PATH < target >
It will be interesting to understand why it works in linux
Recently started with Calabash on iOS.
I have a simple code that does a touch("label marked:'Username'") and then uses keyboard_enter_text to enter the text, but for some reason when I run this via cucumber from command line or without setting breakpoints in Rubymine the code just runs and does not produce the desired result and the touch does not seem to do anything and the keyboard is not activated. I am doing this on a real device, (I have tried various devices).
How ever if I debug the code using ruby mine using breakpoint it produces the desired result.
I have the below setup
$ xcode-select --print-path
/Applications/Xcode.app/Contents/Developer
$ xcodebuild -version
Xcode 6.1.1
Build version 6A2008a
$ calabash-ios version
0.11.4
curl http://:37265/version
{"app_version":"4.3",
"outcome":"SUCCESS",
"app_id":"****",
"simulator_device":"",
"version":"0.11.4",
"app_name":"***",
"iphone_app_emulated_on_ipad":false,
"4inch":true,
"git":{"remote_origin":"git#github.com:calabash\/calabash-ios-server.git",
"branch":"master","revision":"bcc992b"},
"screen_dimensions":{"scale":2,"width":640,"sample":1,"height":1136},
"iOS_version":"7.1.2",
"system":"iPhone5,2","simulator":""}
I have checked that My App is linked with the correct calabash.framework and that it matches the gem version 0.11.4
When I debug using ruby mine then the steps run successfully (the touch works and the keyboard is activated), how ever when I run from command line or run without breakpoints in Ruby mine, the touch just seems to fail without issuing any warnings or errors. Not sure what is causing it, I have upgraded from Xcode 6.1 to Xcode 6.1.1 and that too did not help.
I have noticed similar issues reported by people in the past, but they have been around older versions of the calabash iOS gem and supposed to have been fixed in 0.11.4
When I run the app in DEBUG mode with DEBUG=1 and HTTP_DEBUG=1 I can see below
Command line run in which touch does not do anything
`
! CONNECT TO <ip>:37265
! CONNECTION ESTABLISHED
POST /map HTTP/1.1
Content-Type: application/x-www-form-urlencoded
User-Agent: HTTPClient/1.0 (2.3.4.1, ruby 2.1.5 (2014-11-13))
Accept: */*
Date: Fri, 27 Feb 2015 16:19:55 GMT
Content-Length: 97
Host: <ip>:37265
{"query":"view marked:'Problems signing in?'","operation":{"method_name":"query","arguments":[]}}
= Response
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Length: 66
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
Date: Fri, 27 Feb 2015 16:19:56 GMT
Accept-Ranges: bytes
Access-Control-Max-Age: 3000
{"status_bar_orientation":"down","results":[],"outcome":"SUCCESS"}
`
Debug run from ruby mine using breakpoint
CONNECT TO <ip>:37265
! CONNECTION ESTABLISHED
POST /map HTTP/1.1
Content-Type: application/x-www-form-urlencoded
User-Agent: HTTPClient/1.0 (2.3.4.1, ruby 1.9.3 (2013-06-27))
Accept: */*
Date: Fri, 27 Feb 2015 16:18:07 GMT
Content-Length: 97
Host: <ip>:37265
{"query":"view marked:'Problems signing in?'","operation":{"method_name":"query","arguments":[]}}
= Response
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Length: 478
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
Date: Fri, 27 Feb 2015 16:18:08 GMT
Accept-Ranges: bytes
Access-Control-Max-Age: 3000
{"status_bar_orientation":"down","results":[{"text":"Problems signing in?","enabled":true,"rect":{"center_x":160,"y":504,"width":200,"x":60,"center_y":526,"height":44},"id":null,"description":"<UILabel: 0x14dde8d0; frame = (60 504; 200 44); text = 'Problems signing in?'; gestureRecognizers = <NSArray: 0x14dde1f0>; layer = <CALayer: 0x14dde980>>","label":"Problems signing in?","alpha":1,"class":"UILabel","frame":{"y":504,"width":200,"x":60,"height":44}}],"outcome":"SUCCESS"}= Request
`
As we can see the debug run in Rubymine seems to return information about frame ordinates that are missing in the first run
The log output at /var/folders/25/9_sb609576jcmgdbqt553ps08n5b5c/T/run_loop20150227-8208-1g7ccvv/run_loop.out for the failed run
2015-02-27 16:19:51.871 instruments[8229:591f] WebKit Threading Violation - initial use of WebKit from a secondary thread.
2015-02-27 16:19:56 +0000 Default: OUTPUT_JSON:
{"status":"success","value":true,"index":0}
END_OUTPUT
And for the successful debug run (that activated the touch and the keyboard) at /var/folders/25/9_sb609576jcmgdbqt553ps08n5b5c/T/run_loop20150227-7941-1c6mznp/run_loop.out
2015-02-27 16:18:00.085 instruments[7962:4f17] WebKit Threading Violation - initial use of WebKit from a secondary thread.
2015-02-27 16:18:04 +0000 Default: OUTPUT_JSON:
{"status":"success","value":true,"index":0}
END_OUTPUT
The run_loop.out does not seem to give any indication either.
Am I missing something ? Why does the touch not have any effect and not activate the keyboard. Any help is appreciated.
I have a couple of suggestions. These are not definitive answers, but the text is too long for a comment.
I have noticed similar issues reported by people in the past, but they have been around older versions of the calabash iOS gem and supposed to have been fixed in 0.11.4
Can you update your gem and server to the latest 0.13.0? There have been many fixes since 0.11.4.
Is it possible that you are seeing this: NSLog output can cause apps to become unresponsive during testing.?
works when debugging
I suspect this is because of timing issue, but I cannot confirm until I see some Steps or a list of commands executed in the irb.