[gpadmin#mdw ssb_gp_scripts]$ cat d_gpload.yaml
—
VERSION: 1.0.0.1
DATABASE: ssb
USER: gpadmin
HOST: mdw
PORT: 5432
GPLOAD:
INPUT:
– SOURCE:
LOCAL_HOSTNAME:
– mdw
PORT: 8080
FILE:
– /ssb/ssb/dimdate.tbl
SSL: false
– FORMAT: csv
– DELIMITER: ‘|’
– HEADER: false
– ENCODING: UNICODE
– ERROR_LIMIT: 100
– LOG_ERRORS: true
EXTERNAL:
– SCHEMA: orders
OUTPUT:
– TABLE: orders.dimdate
– MODE: insert
PRELOAD:
– TRUNCATE: true
– REUSE_TABLES: true
Above is the yaml file on master host.
[gpadmin#mdw ssb_gp_scripts]$ gpload -f d_gpload.yaml -l d_gpload.log .
2018-10-13 13:12:02|INFO|gpload session started 2018-10-13 13:12:02
2018-10-13 13:12:02|INFO|started gpfdist -p 8080 -P 8081 -f “/ssb/ssb/dimdate.tbl” -t 30
2018-10-13 13:12:02|INFO|reusing external table ext_gpload_reusable_179f5634_ced8_11e8_822a_0a78550cb23a
It hangs at this point and never moves.
My cluster is in AWS.
The issue was due to HTTP port 8080 not being open between the nodes.
Opened up this port in AWS Security Group and the issue was resolved.
Related
Well, before I asked this question, I searched a lot and couldn't find a proper answer (or the question was wrongly asked).
I want to connect a bridge to a GRE tunnel on netplan.
I can successfully do the connection, but the routing table doesn't get updated correctly. I am forced to add the route manually for it to work.
So, here's my netplan setup:
network:
version: 2
ethernets:
enp1s0f0:
dhcp4: no
accept-ra: false
addresses:
- 192.168.0.100/24
routes:
- to: default
via: 192.168.0.254
match:
macaddress: d8:5e:d3:43:cd:ae
set-name: enp1s0f0
nameservers:
addresses:
- 1.1.1.1
- 1.0.0.1
- 2606:4700:4700::1111
- 2606:4700:4700::1001
tunnels:
gre1:
mode: gre
remote: 192.168.100.150
local: 192.168.0.100
bridges:
br1:
dhcp4: false
dhcp6: false
optional: true
interfaces: [ gre1 ]
addresses:
- 172.16.20.2/30
routes:
- to: default
via: 172.16.20.1
scope: link
table: 100
routing-policy:
- from: 172.16.20.0/30
table: 100
With the above configuration, the gre1 has a state UNKNOWN and the br1 state is DOWN.
If I try to ping 172.16.20.1 it fails.
But, if I run:
ip route add 172.16.20.0/30 dev gre1
I can ping without any issues.
Anyone has a clue on how to solve this riddle?
So I have a container running with port forwarding set up. It seems that the port is listening on the local windows host, for some reason, the connection won't go through.
The command to run the docker container:
docker run -p 4400:4400 storybook:latest
Inside the container itself, I can verify the service is running on port 4400:
netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:4400 0.0.0.0:* LISTEN 33/node
wget http://0.0.0.0:4400
--2022-08-23 19:57:12-- http://0.0.0.0:4400/
Connecting to 0.0.0.0:4400... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5866 (5.7K) [text/html]
Saving to: 'index.html'
100%[==============================================================================>] 5,866 --.-K/s in 0s
2022-08-23 19:57:12 (363 MB/s) - 'index.html' saved [5866/5866]
And on the windows host, I can verify docker is listening on port 4440:
netstat -aon | find /i "listening"
TCP [::]:4400 [::]:0 LISTENING 20412
tasklist
Image Name PID Session Name Session# Mem Usage
========================= ======== ================ =========== ============
com.docker.backend.exe 20412 Console 1 26,540 K
But I can't access the service via the Windows host.
wget http://localhost:4400
wget : The underlying connection was closed: The connection was closed unexpectedly.
I even tried getting the IP address of the docker container:
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
898079f335be storybook:latest "npx nx serve storyb…" 34 minutes ago Up 34 minutes 0.0.0.0:4400->4400/tcp, :::4400->4400/tcp relaxed_mayer
> docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' 898079f335be
172.17.0.2
And tried accessing the service via that IP:
wget http://172.17.0.2:4400
wget : Unable to connect to the remote server
The version of windows:
Edition: Windows 10 Enterprise
Version: 21H2
OS Build: 19044.1766
Docker information:
Client:
Context: desktop-linux
Debug Mode: false
Plugins:
buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
compose: Docker Compose (Docker Inc., 2.0.0-beta.4)
scan: Docker Scan (Docker Inc., v0.8.0)
Server:
Containers: 7
Running: 1
Paused: 0
Stopped: 6
Images: 22
Server Version: 20.10.7
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.4.72-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 25.06GiB
Name: docker-desktop
ID: VFD3:RX76:D4JD:5Z6P:R2IQ:7JD4:FFQS:YDLJ:BDNW:J4UX:4U5A:GF4S
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
EDIT: I am using WSL2 as the backend.
I have a ubuntu remote machine which I access using ssh and I have installed mongodb on it, and I wish to access it remotely using the uri of the mongodb which is
mongodb://<machine_ip_address>:27017/<database_name>
I have also tried to change the /etc/mongod.conf and the edited file is this:
systemLog:
destination: file
path: "/var/log/mongodb/mongod.log"
logAppend: true
storage:
journal:
enabled: true
processManagement:
fork: true
net:
bindIp: 127.0.0.1,my_ip
port: 27017
setParameter:
enableLocalhostAuthBypass: false
Is there a way I can give my local machine access to the remote server db by entering the uri of the remote server mongodb in my spring boot application?
I am trying to set up automated integration tests against an Oracle database and am
planning on using https://www.testcontainers.org/ to
launch the containers in my docker-compose.yml.
I have succeeded in getting Dockerized Oracle running on my laptop
using these instructions:
https://github.com/oracle/docker-images/tree/master/OracleWebLogic/samples/12212-oradb-wlsstore
The steps are:
# see https://docs.oracle.com/cd/E37670_01/E75728/html/oracle-registry-server.html
docker login container-registry.oracle.com # supply your oracle credentials as described above
docker network create -d bridge SampleNET
cat > env.txt <<EOF
DB_SID=InfraDB
DB_PDB=InfraPDB1
DB_DOMAIN=us.oracle.com
DB_BUNDLE=basic
EOF
docker run -d --name InfraDB --network=SampleNET -p 1521:1521 -p 5500:5500 --env-file env.txt -it --shm-size="8g" store/oracle/database-enterprise:12.2.0.1
After a minute when I issue the command "sudo docker ps -a" I see the DB is up and healthy.
CONTAINER ID IMAGE COMMAND
6a276a311b2e store/oracle/database-enterprise:12.2.0.1 "/bin/sh -c '/bin/ba…"
CREATED STATUS PORTS
2 minutes ago Up 2 minutes (healthy) 0.0.0.0:1521->1521/tcp, 0.0.0.0:5500->5500/tcp
NAMES
InfraDB
And I can now run sqlplus (from a shell prompt on my laptop, not a Docker container):
sqlplus sys/Oradoc_db1#localhost:1521/InfraDB.us.oracle.com AS SYSDBA
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
SQL>
My goal now is to create a Docker compose file that allows me to
the same thing as what was accomplished with the command line invocation of Dockerized
Oracle. Here is the docker-compose.yml file I tried:
version: '2'
services:
database:
image: store/oracle/database-enterprise:12.2.0.1
ports:
- 1521:1521
- 8080:8080
- 5500:5500
environment:
- DB_SID:InfraDB
- DB_PDB:InfraPDB1
- DB_DOMAIN:us.oracle.com
- DB_BUNDLE:basic
networks:
default:
external:
name: SampleNET
So, with this docker compose file I can launch a shell in the
container running my Oracle DB and run sqlplus:
container=`sudo docker ps -a | grep oracle | sed -e's/\s.*//'`
sudo docker exec -t -i $container bash
# SESSION INSIDE DOCKER CONTAINER:
[oracle#8f26f224db03 /]$ sqlplus sys/Oradoc_db1 AS SYSDBA
SQL*Plus: Release 12.2.0.1.0 Production on Tue Mar 3 02:44:03 2020
Copyright (c) 1982, 2016, Oracle. All rights reserved.
Now, the million dollar question:
What is the syntax of the sqlplus command I would specify
to connect FROM MY LAPTOP (i.e., not launching a shell in the
container running the DB server)?
Here are some of the many variants I tried (note: I tried with localhost, 127.0.0.1, and
the actual IP of my laptop):
sqlplus sys/Oradoc_db1#'(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=localhost)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=InfraDB)))'
sqlplus sys/Oradoc_db1#'(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=InfraDB.us.oracle.com)))'
Diagnostic Note:
I am running Docker on Mac.
After I launch Docker-ized Oracle I do see listeners on the ports that the docker compose configuration is forwarding, as shown below:
sudo lsof -i -P | grep -i "listen" | egrep "1521|8080|5500"
com.docke 6295 christopher.bedford 33u IPv6 0xfffffffffffffff 0t0 TCP *:1521 (LISTEN)
com.docke 6295 christopher.bedford 34u IPv6 0xfffffffffffffff 0t0 TCP *:5500 (LISTEN)
com.docke 6295 christopher.bedford 38u IPv6 0xfffffffffffffff 0t0 TCP *:8080 (LISTEN)
Problem Solved
Here is a summary of what I did to solve the problem thanks to the suggestion of my friend Matt (Changos Muertos below).
1). creation of the bridge network 'SampleNET' is not required.
2). seems like specifying environment variables inline like this (below) does not work
environment:
- DB_SID:InfraDB
- DB_PDB:InfraPDB1
- DB_DOMAIN:us.oracle.com
- DB_BUNDLE:basic
3). Instead it is better to define those environment settings in a file that is referenced by the 'env_file' configuration option as shown below. This file resides on /tmp/docker-compose.yml
version: '2'
services:
database:
image: store/oracle/database-enterprise:12.2.0.1
ports:
- 1521:1521
- 8080:8080
- 5500:5500
env_file:
- e2.txt
And the env file contents live in the same directory ( /tmp/e2.txt ), and has the content below:
DB_SID=InfraDB
DB_PDB=InfraPDB1
DB_DOMAIN=us.oracle.com
DB_BUNDLE=basic
To run i do
docker-compose -f /tmp/docker-compose.yml up
and then i can connect (from my laptop as was the original goal) via:
sqlplus sys/Oradoc_db1#0.0.0.0:1521/InfraDB.us.oracle.com as SYSDBA
This depends on a few things, like network type, and if the port is forwarded properly. Then verify this on the host (in linux 'netstat -tlpn' ), this will show you if it is listening on the port, and if so what interface and protocol. The tool may differ depending on the host OS.
Once you see it listening via tcp (not tcp6 only), and the firewall is open on the host, you can combine that info to make your connection string.
I just want to run packetbeat and get packet sniff from MySQL and output to file or console ,so that I no need Elastic system
I tried to run it but no thing output
root#localhost~: packetbeat -c packetbeat.yml
root#localhost~:
Following are my config file
procs:
enabled: true
monitored:
- process: mysqld
cmdline_grep: mysqld
output:
### Console output
console:
# Pretty print json event
pretty: false
How can I do that ?
Packetbeat works by capturing the network traffic that Mysql creates, so you need to also configure from which device to capture the traffic and on which tcp ports Mysql is running. For example:
interface:
device: any
protocols:
mysql:
ports: [3306]
procs:
enabled: true
monitored:
- process: mysqld
cmdline_grep: mysqld
output:
### Console output
console:
# Pretty print json event
pretty: false
Your console output configuration looks good to me. You can also output to rotating files, if you prefer.