数据库学习案例20240306-oracle数据库连接类故障trace

1 数据库连接类故障TRACE

故障现象

客户端tnsping ipv6:1521 提示Permission denied，但是在本机是可以链接的。

查看数据库netstat -tunlp|grep 1521

TCP 地址：1521

TCP 不全的IPV6：1521

没有tcp6协议出现。

故障分析

客户端sqlnet.ora 添加如下内容进行tnsping

Trace_level_client=16
Trace_directory_client=/home/oracle/
Trace_unique_client=on
Trace_timestamp_client=on
Diag_adr_enabled=off
tnsping.trace_directory=/home/oracle/
tnsping.trace_level=admin

#TRACE_LEVEL_SERVER=16
#TRACE_FILE_SERVER=server_trace
#TRACE_DIRECTORY_SERVER=/ogg

查看trc文件

[oracle@orcl-jiekou24 ~]$ more tnsping.trc

TNS Ping Utility for Linux: Version 11.2.0.4.0 - Production on 06-MAR-2024 22:53:26

--- TRACE CONFIGURATION INFORMATION FOLLOWS ---
New trace stream is /home/oracle/tnsping.trc
New trace level is 6
--- TRACE CONFIGURATION INFORMATION ENDS ---
--- PARAMETER SOURCE INFORMATION FOLLOWS ---
Attempted load of system pfile source /u01/app/oracle/product/11.2.0/db_1/network/admin/sqlnet.ora
Parameter source loaded successfully

-> PARAMETER TABLE LOAD RESULTS FOLLOW <-
Successful parameter table load
-> PARAMETER TABLE HAS THE FOLLOWING CONTENTS <-
Diag_adr_enabled = off
tnsping.trace_level = admin
tcp.validnode_checking = yes
tcp.invited_nodes = (10.36.246.176,10.36.246.178,10.36.246.179,10.36.246.180,10.36.246.181,10.36.246.182,10.36.224.0/24,10.36.127.116,10.36.127.117,10.36.245.0/24,10.36.233.91,10.36.201.25,10.36.
60.208,10.36.60.0/24,10.36.200.0/22,10.36.248.0/24,10.36.193.0/24,10.143.13.0/24,1405:8002:5a06:120:10:0:2:27)
Trace_level_client = 16
tnsping.trace_directory = /home/oracle/
Trace_unique_client = on
sqlnet.inbound_connect_timeout = 30
Trace_directory_client = /home/oracle/
Trace_timestamp_client = on
--- PARAMETER SOURCE INFORMATION ENDS ---
--- LOG CONFIGURATION INFORMATION FOLLOWS ---
Log stream will be "standard output"
Log stream validation not requested
--- LOG CONFIGURATION INFORMATION ENDS ---

nlstdipi: entry
nlstdipi: exit
nnfun2awanm: entry
nnfgiinit: entry
nncpcin_maybe_init: default name server domain is [root]
nnfgiinit: Installing read path
nnfgsrsp: entry
nnfgsrsp: Obtaining path parameter from names.directory_path or native_names.directory_path
nnfgsrsp: Parmeter names.directory_path not found, setting path to compiled in default
nnfgspd: entry
nnfgspd: setting element TNSNAMES
nnfgspd: setting element HOSTNAME
nnfgspd: setting element EZCONNECT
nnfgspd: setting element LDAP
nnfgspd: Path set
nnfgspd: exit
nnfun2a: entry
nlolgobj: entry
nnfgrne: entry
nnfgrne: Going though read path adapters
nnfgrne: Switching to TNSNAMES adapter
nnftboot: entry
nlpaxini: entry
nlpaxini: exit
nnftmlf_make_local_addrfile: entry
nnftmlf_make_local_addrfile: construction of local names file failed
nnftmlf_make_local_addrfile: exit
nlpaxini: entry
nlpaxini: exit
nnftmlf_make_system_addrfile: entry
nnftmlf_make_system_addrfile: system names file is /u01/app/oracle/product/11.2.0/db_1/network/admin/tnsnames.ora
nnftmlf_make_system_addrfile: exit
nnftboot: exit
nnftrne: entry
nnftrne: Original name: orcl6
nnfttran: entry
nnfttran: exit
nnftrne: Using tnsnames.ora address (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST =1405:8002:5a06:120:10:0:2:605b)(PORT = 1521))) (CONNECT_DATA = (SERVICE_NAME = orcl))) for name orcl6
nnftrne: exit
nnfgrne: exit
nlolgserv: entry
nnfggav: entry
nnftgav: entry
nnftgav: exit
nnfgfrm: entry
nnftfrm: entry
nnftfrm: exit
nnfgfrm: exit
nlolgserv: exit
nlolgobj: exit
nlolfmem: entry
nlolfmem: exit
nnfun2awanm: Getting the path of sqlnet.ora
nnfun2awanm: Getting the adapter name
nnfun2awanm: exit
snsgblini: exit
nsmal: 272 bytes at 0x1494070
nscall: connecting...
snlinGetAddrInfo: entry
snlinGetAddrInfo: exit
snlinFreeAddrInfo: entry
snlinFreeAddrInfo: exit
nladini: entry
nladini: exit
nladget: entry
nladget: exit
nsmal: 123 bytes at 0x14ab6b0
nsc2addr: (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=1405:8002:5a06:120:10:0:2:605b)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=orcl)))
nttbnd2addr: entry
snlinGetAddrInfo: entry
snlinGetAddrInfo: exit
nttbnd2addr: using host IP address: 1405:8002:5a06:120:10:0:2:605b
snlinFreeAddrInfo: entry
snlinFreeAddrInfo: exit
nttbnd2addr: exit
nsmal: 1576 bytes at 0x14ab740
nsmal: 2760 bytes at 0x14abd70
nsmal: 168 bytes at 0x14aca40
nsopen: opening transport...
nttcon: entry
nttcon: toc = 1
nttcnp: entry
nttcnp: exit
nttcni: entry
nttcni: Tcp conn timeout = 60000 (ms)
nttctl: entry
nttctl: Setting connection into non-blocking mode
nttcni: trying to connect to socket 5.
ntt2err: entry
ntt2err: exit
ntctst: size of NTTEST list is 1 - not calling poll
sntpoltst: exit
ntt2err: entry
ntt2err: soc 5 error - operation=1, ntresnt[0]=516, ntresnt[1]=13, ntresnt[2]=0
ntt2err: exit
nttcni: exit
nttcon: exit
nserror: nsres: id=0, op=65, ns=12546, ns2=12560; nt[0]=516, nt[1]=13, nt[2]=0; ora[0]=0, ora[1]=0, ora[2]=0
nsopen: unable to open transport
nsvntx_dei: entry
nsvntx_dei: exit
nsmfr: 2760 bytes at 0x14abd70
nsmfr: 1576 bytes at 0x14ab740
nsmfr: 123 bytes at 0x14ab6b0
nscall: connecting...
nladget: entry
nladget: exit
nsmfr: 272 bytes at 0x1494070
nladtrm: entry
nladtrm: exit
nlse_term_audit: entry
nlse_term_audit: exit

发现在数据传输出现问题。
[oracle@orcl-jiekou24 ~]$ ping6 1405:8002:5a06:120:10:0:2:605b
PING 1405:8002:5a06:120:10:0:2:605b(1405:8002:5a06:120:10:0:2:605b) 56 data bytes
64 bytes from 1405:8002:5a06:120:10:0:2:605b: icmp_seq=1 ttl=60 time=0.316 ms

64 bytes from 1405:8002:5a06:120:10:0:2:605b: icmp_seq=2 ttl=60 time=0.287 ms
64 bytes from 1405:8002:5a06:120:10:0:2:605b: icmp_seq=3 ttl=60 time=0.392 ms
^C
--- 1405:8002:5a06:120:10:0:2:605b ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.287/0.331/0.392/0.049 ms
[oracle@orcl-jiekou24 ~]$
[oracle@orcl-jiekou24 ~]$ telnetr
bash: telnetr: command not found...
Similar command is: 'telnet'
[oracle@orcl-jiekou24 ~]$ telnet
telnet> ^C
[oracle@orcl-jiekou24 ~]$
[oracle@orcl-jiekou24 ~]$ telnet 1405:8002:5a06:120:10:0:2:605b 152
Trying 1405:8002:5a06:120:10:0:2:605b...
telnet: connect to address 1405:8002:5a06:120:10:0:2:605b: Permission denied
[oracle@orcl-jiekou24 ~]$ telnet 1405:8002:5a06:120:10:0:2:605b 1521
Trying 1405:8002:5a06:120:10:0:2:605b...
telnet: connect to address 1405:8002:5a06:120:10:0:2:605b: Permission denied
[oracle@orcl-jiekou24 ~]$ telnet 10.36.246.91 1521
Trying 10.36.246.91...
Connected to 10.36.246.91.
Escape character is '^]'.
^C^CConnection closed by foreign host.
[oracle@orcl-jiekou24 ~]$ telnet 10.36.246.91 1521
Trying 10.36.246.91...
Connected to 10.36.246.91.
Escape character is '^]'.
^C^CConnection closed by foreign host.
[oracle@orcl-jiekou24 ~]$
[oracle@orcl-jiekou24 ~]$
[oracle@orcl-jiekou24 ~]$ telnet 1405:8002:5a06:120:10:0:2:605b 1521
Trying 1405:8002:5a06:120:10:0:2:605b...
telnet: connect to address 1405:8002:5a06:120:10:0:2:605b: Permission denied
[oracle@orcl-jiekou24 ~]$

换一台正式数据库

[patrol@orcl-jiekou24 ~]$ telnet 1405:8002:5A06:0120:0010:0000:0002:D00C 1521
Trying 1405:8002:5a06:120:10:0:2:d00c...
Connected to 1405:8002:5A06:0120:0010:0000:0002:D00C.
Escape character is '^]'.
^C^CConnection closed by foreign host.
[patrol@orcl-jiekou24 ~]$

sysctl -a|grep ipv6

就是没有TCP6:

问题处理

问题处理可以参考metalink文档

ORA-12546 TNS Permission Denied with a remote Client (Doc ID 2420034.1)

What is happening is this:
- Oracle supplies the "address" information (protocol, host, and port) to the Operating System, which in turn gets a TCP
Socket from the Network Stack.
2024/3/6 23:22 Document Display
https://support.oracle.com/epmos/faces/SearchDocDisplay?_adf.ctrl-state=byhznok0_4#CAUSE 3/3
- Oracle sits and waits for the "approval" and a Socket Number.
- In this case, there is an immediate OS error when a Socket should be provided:
ntresnt[1]=13
Unfortunately, this is all external to Oracle and happening at the OS / Network level.
As we are a guest of the Network and System, we have no settings that can "over ride" or workaround such mandatory
communication requirements.
So you need to ask your own Systems / Network Admin to do some in depth analysis of the TCP stack (especially what is
set for limiting remote TCP communications) and find out what is preventing this TCP communication on this address.
Some ideas (again at the Network level):
- check that the hostname(s) and IP address(es) are not "plumbed" (linked) to multiple values.
- check that both lookup and reverse lookup is working.