hCatalog page gives error - hortonworks-data-platform

I am using HortonWorks sandbox to try out few samples. Following page is displaying "Error" on UI (Time Out) http://:8000/hcatalog/
Detailed Server logs:
[25/Apr/2014 13:07:49 +0000] middleware INFO Processing exception: timed out (code THRIFTSOCKET): None:
Traceback (most recent call last):
File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.2.3-py2.6.egg/djang‌​o/core/handlers/base.py", line 100, in get_response response = callback(request, *callback_args, **callback_kwargs)
Any advice?

I suspect this is due to memory. Your memory should be at least 4096 MB.

Related

Oracle BI Start up fails to start Admin Server

I'm trying to startup OBIEE 12c
Database is up and running
But running the start.cmd fails when starting Admin Server with this error message.
"""""
Starting AdminServer ...
Unable to connect to AdminServer on host: ***********
Failed to start one or more Servers
/Servers/AdminServer/ListenPort=9500
Accessing admin server using URL t3://*************
Start Admin Server connect Exception caught Error occurred while performing connect : Error getting the initial context. There is no server running at t3://************* : Failed to initialize JNDI context, tried 2 time or times totally, the interval of each time is 0ms.
t3://*************: Destination **********, 9500 unreachable.; nested exception is:
java.net.ConnectException: Connection refused: connect; No available router to destination.; nested exception is:
java.rmi.ConnectException: No available router to destination.
Use dumpStack() to view the full stacktrace :
Reading domain...
Error: runCmd() failed. Do dumpStack() to see details.
Failed to get Status of Servers and System Components
"""""
Start.cmd error message
Also, trying to check the status come up with this error.
"""""
Start Admin Server connect Exception caught Error occurred while performing connect : Error getting the initial context. There is no server running at t3://************* : Failed to initialize JNDI context, tried 2 time or times totally, the interval of each time is 0ms.
t3://*************: Destination 10.10.3.88, 9500 unreachable.; nested exception is:
java.net.ConnectException: Connection refused: connect; No available router to destination.; nested exception is:
java.rmi.ConnectException: No available router to destination.
Use dumpStack() to view the full stacktrace :
Reading domain...
Error: runCmd() failed. Do dumpStack() to see details.
Problem invoking WLST - Traceback (innermost last):
File "F:\Middleware\bi\modules\oracle.bi.sysman\scripts\status_servers.py", line 29, in ?
File "F:\Middleware\bi\modules\oracle.bi.sysman\scripts\process_control.py", line 581, in statusComponents
File "F:\Middleware\bi\modules\oracle.bi.sysman\scripts\process_control.py", line 455, in outputComponentsStatus
File "F:\Middleware\bi\modules\oracle.bi.sysman\scripts\process_control.py", line 243, in connectAdminServer
File "F:\Middleware\bi\modules\oracle.bi.sysman\scripts\process_control.py", line 179, in requestCredentialsAndConnectToAdminServer
File "F:\Middleware\bi\modules\oracle.bi.sysman\scripts\process_control.py", line 513, in getAdminServerUrl
File "C:\Users\Administrator\AppData\Local\Temp\1\WLSTOfflineIni5666259588766029647.py", line 131, in readDomain
File "C:\Users\Administrator\AppData\Local\Temp\1\WLSTOfflineIni5666259588766029647.py", line 19, in command
60713: Attempt to execute command "readDomain" in invalid state: Configuration
"""""
STATUS ERROR MESSAGE

Error: Cross origin localhost forbidden on Ajax call ScalaJS

I'm trying to perform a simple "GET" on wikipedia:
Ajax.get("https://en.wikipedia.org/w/index.php?title=Special:Random") map (_.responseText)
I get this error when I try to run:
[error] org.scalajs.jsenv.ExternalJSRun$NonZeroExitException: exited with code 1
[error] at org.scalajs.jsenv.ExternalJSRun$$anon$1.run(ExternalJSRun.scala:186)
[error] stack trace is suppressed; run 'last Compile / run' for the full output
[error] (Compile / run) org.scalajs.jsenv.ExternalJSRun$NonZeroExitException: exited with code 1
[error] Total time: 3 s, completed Mar 4, 2021, 4:47:42 PM
Error: Cross origin http://localhost forbidden
at dispatchError (C:\Users\mjwin\IdeaProjects\wiki-brain\node_modules\jsdom\lib\jsdom\living\xhr\xhr-utils.js:62:19)
at Object.validCORSHeaders (C:\Users\mjwin\IdeaProjects\wiki-brain\node_modules\jsdom\lib\jsdom\living\xhr\xhr-utils.js:74:5)
at Request.<anonymous> (C:\Users\mjwin\IdeaProjects\wiki-brain\node_modules\jsdom\lib\jsdom\living\xhr\XMLHttpRequest-impl.js:673:25)
at Request.emit (events.js:315:20)
at Redirect.onResponse (C:\Users\mjwin\IdeaProjects\wiki-brain\node_modules\request\lib\redirect.js:147:11)
at Request.onRequestResponse (C:\Users\mjwin\IdeaProjects\wiki-brain\node_modules\request\request.js:986:22)
at ClientRequest.emit (events.js:315:20)
at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:641:27)
at HTTPParser.parserOnHeadersComplete (_http_common.js:126:17)
at TLSSocket.socketOnData (_http_client.js:509:22) {
type: 'XMLHttpRequest'
}
I've looked around for info on this error, but I'm still not sure what it means. I think I understand that I'm running into a CORS issue, but I don't understand what to do about it.
You need allowed to read a file into a URL:
Cross-Origin Read Blocking (CORB) blocked cross-origin response
Without allowing your url to access. Your script got an error.

Ambari fails restarting : The package hive-metastore is not supported by this version of the stack-select tool

I build a hadoop cluster with Ambari 2.7.3.0. Ambari notifies we need to restart some components, but I get the following error message when I restart them.
I guess there are some version issues as the message mentions.
Fail: The package hive-metastore is not supported by this version of the stack-select tool.
Is it due to the limited connection to the internet? Because of security issues, we use a local repository.
Unfortunately, I'm very new to Ambari and hadoop, so I cannot find any clue in this message.
2019-03-14 13:20:46,629 - Reporting component version failed
Traceback (most recent call last):
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 363, in execute
self.save_component_version_to_structured_out(self.command_name)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 223, in save_component_version_to_structured_out
stack_select_package_name = stack_select.get_package_name()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 109, in get_package_name
package = get_packages(PACKAGE_SCOPE_STACK_SELECT, service_name, component_name)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 234, in get_packages
raise Fail("The package {0} is not supported by this version of the stack-select tool.".format(package))
Fail: The package hive-metastore is not supported by this version of the stack-select tool.
2019-03-14 13:20:46,676 - Reporting component version failed
Traceback (most recent call last):
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 363, in execute
self.save_component_version_to_structured_out(self.command_name)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 223, in save_component_version_to_structured_out
stack_select_package_name = stack_select.get_package_name()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 109, in get_package_name
package = get_packages(PACKAGE_SCOPE_STACK_SELECT, service_name, component_name)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 234, in get_packages
raise Fail("The package {0} is not supported by this version of the stack-select tool.".format(package))
Fail: The package hive-metastore is not supported by this version of the stack-select tool.

Bizarre Exception from PySpark of Local Mode in Docker Container

I have a Spark application inside a test.py that I want to execute. Briefly speaking, I had Spark 2.3.0 installed, then would like to execute test.py. When I did it on my developing machine, Mac Book, everything is fine. BUT when attempting to do the same thing in a Docker container on my Mac Book, I would encounter the following exception which I've searched for possible hints from Google without triumph.
Clues Collected So Far
Running test.py without Docker on my laptop would be absolutely fine.
It seems to have something to do with record numbers when saving.
I. There are three actions where all have similar logic. In the Docker container, the first action is executed very well. But the next two would fail EVERY TIME when Spark tried to save results to AWS S3.
II. If I limit the dataframes as output in the last 2 actions like 5000, the program would be executed very well.
Logic of Spark Application
Retrieve data from MySQL or S3, do some computation, then store result onto AWS S3.
Exception Message
Exception happened during processing of request from ('127.0.0.1', 52218)
2018-04-24 06:17:19,572 678 py4j.java_gateway INFO:Error while receiving.
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 1062, in send_command
raise Py4JNetworkError("Answer from Java side is empty")
py4j.protocol.Py4JNetworkError: Answer from Java side is empty
2018-04-24 06:17:19,626 678 root ERROR:Exception while sending command.
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 1062, in send_command
raise Py4JNetworkError("Answer from Java side is empty")
py4j.protocol.Py4JNetworkError: Answer from Java side is empty
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 908, in send_command
response = connection.send_command(command)
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 1067, in send_command
"Error while receiving", e, proto.ERROR_ON_RECEIVE)
py4j.protocol.Py4JNetworkError: Error while receiving
Traceback (most recent call last):
Traceback (most recent call last):
File "py/apmain.py", line 36, in <module>
File "/usr/lib64/python3.6/socketserver.py", line 317, in _handle_request_noblock
self.process_request(request, client_address)
File "/usr/lib64/python3.6/socketserver.py", line 348, in process_request
self.finish_request(request, client_address)
File "/usr/lib64/python3.6/socketserver.py", line 361, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib64/python3.6/socketserver.py", line 696, in __init__
self.handle()
File "/opt/spark/python/pyspark/accumulators.py", line 235, in handle
num_updates = read_int(self.rfile)
File "/opt/spark/python/pyspark/serializers.py", line 685, in read_int
raise EOFError
EOFError
----------------------------------------
main()
File "py/apmain.py", line 30, in main
engine_main.main(sys.argv)
File "/opt/ap2126/py/dtt/ml/framework/engine_main.py", line 46, in main
raise e
File "/opt/ap2126/py/dtt/ml/framework/engine_main.py", line 36, in main
engine.execute(tags)
File "/opt/ap2126/py/dtt/ml/framework/engine.py", line 152, in execute
passing_datas = layer_handler.execute(passing_datas, tags, is_load_cache=True)
File "/opt/ap2126/py/dtt/ml/framework/layer_handler.py", line 319, in execute
is_load_cache)
File "/opt/ap2126/py/dtt/ml/framework/layer_handler.py", line 87, in _execute_workers_as_sequence
output_data = worker_handler.worker.do_job(input_datas, results)
File "/opt/ap2126/py/ap2126/label_extraction.py", line 498, in do_job
self._get_lbls_from_structured_job('mysql')
File "/opt/ap2126/py/ap2126/label_extraction.py", line 583, in _get_lbls_from_structured_job
.json(job_save_temp_path)
File "/opt/spark/python/pyspark/sql/readwriter.py", line 775, in json
self._jwrite.json(path)
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 1160, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/spark/python/pyspark/sql/utils.py", line 63, in deco
return f(*a, **kw)
File "/usr/lib/python3.6/site-packages/py4j/protocol.py", line 328, in get_return_value
format(target_id, ".", name))
py4j.protocol.Py4JError: An error occurred while calling o818.json
2018-04-24 06:17:23,151 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,152 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,153 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,154 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,155 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,156 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,156 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,158 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,159 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,159 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,216 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,217 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,218 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,219 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,220 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,222 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
2018-04-24 06:17:23,223 678 py4j.java_gateway ERROR:An error occurred while trying to connect to the Java server (127.0.0.1:38899)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 852, in _get_connection
connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/py4j/java_gateway.py", line 990, in start
self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
Part of Dockerfile
.
.
FROM centos:centos7
.
.
RUN yum -y install https://centos7.iuscommunity.org/ius-release.rpm && \
yum -y install python36u python36u-pip python36u-devel && \
python3.6 -m pip install --upgrade pip && \
echo "export PYTHONIOENCODING=utf-8" >> ~/.bashrc && \
echo "alias python='python3'" >> ~/.bashrc && \
# Installation of the Java Runtime Environment
su -c "yum -y install java-1.8.0-openjdk" && \
yum -y install mlocate; updatedb && \
echo "export JAVA_HOME=\"$(locate bin/java | grep jvm | sed 's+/bin/java++g')\"" >> ~/.bashrc && \
# Installation of Spark for running `pyspark` with success
curl -s http://ftp.twaren.net/Unix/Web/apache/spark/spark-2.3.0/spark-2.3.0-bin-hadoop2.7.tgz | tar -zx -C /opt/ && \
ln -s /opt/spark-2.3.0-bin-hadoop2.7 /opt/spark && \
echo "export SPARK_HOME=/opt/spark >> ~/.bashrc"; source ~/.bashrc && \
echo "export PYTHONPATH="$SPARK_HOME"/python/" >> ~/.bashrc; source ~/.bashrc && \
echo "export PYTHONPATH="$PYTHONPATH":./py/" >> ~/.bashrc && \
echo "export PYSPARK_PYTHON=/usr/bin/python3" >> ~/.bashrc; source ~/.bashrc && \
# For the interaction with AWS S3 on Spark
curl -s http://central.maven.org/maven2/org/apache/hadoop/hadoop-aws/2.7.3/hadoop-aws-2.7.3.jar -o ${SPARK_HOME}/jars/hadoop-aws-2.7.3.jar && \
curl -s http://central.maven.org/maven2/com/amazonaws/aws-java-sdk/1.7.4/aws-java-sdk-1.7.4.jar -o ${SPARK_HOME}/jars/aws-java-sdk-1.7.4.jar
.
.
.
How I Run Docker
docker run -it --rm ${image id}
This is due to docker daemon killing the companion java process of the pyspark process because of memory limitations of the running container. The python counterpart detects the socket close and throws the EOF. The other exceptions are launched when trying to reconnect to the dead process.
You should check your process memory consumption (specially the java process which actually performs most of the spark operations) and adjust memory settings (--memory , --memory-swap) for your requirements when running the container (https://docs.docker.com/config/containers/resource_constraints/).

ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:747) on OS X

I am trying this bit of code.
import pandas as pd
fiddy_states=pd.read_html('https://simple.wikipedia.org/wiki/List_of_U.S._states')
print(fiddy_states[0])
I am getting following error:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect
server_hostname=server_hostname)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 401, in wrap_socket
_context=self, _session=session)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 808, in __init__
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1061, in do_handshake
self._sslobj.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 683, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:747)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/avinashmore/Desktop/Desktop pe Desktop/Programming/Python/Pandas/panda4BuidlingDataset.py", line 10, in <module>
fiddy_states= pd.read_html('https://simple.wikipedia.org/wiki/List_of_U.S._states')
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/io/html.py", line 896, in read_html
keep_default_na=keep_default_na)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/io/html.py", line 733, in _parse
raise_with_traceback(retained)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/compat/__init__.py", line 339, in raise_with_traceback
raise exc.with_traceback(traceback)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:747)>
I have tried following things:
Uninstall and install certifi
and all the other steps mentioned in related answers on stack overflow.
Still unable to fix it.
Please help.
I was having the same problem, and after some searching, i came across this
https://bugs.python.org/issue28150
There is a command-line-runnable script ("/Applications/Python 3.6/Install Certificates.command") that uses pip to install certifi and creates a symlink in the OpenSSL directory to certifi's installed bundle location.
This worked for me, maybe you can take a look.

Resources