Skip to main content

ORA-00800: soft external error arguments

The problem as stated:
After upgrading the Oracle Database from 12c to 19c the following error generated in the alert log file.

Actual Oracle Error Code: ORA-00800
After Upgrading from 12c to 19c ORA-00800: soft external error, arguments occured
Oracle Database Enterprise 19c fails to start with error: ORA-00800: soft external error, arguments: [Set Priority Failed], [VKTM]
Why does Insights prevent Oracle grid services from starting
How to configure the User slice to allow Real-Time Scheduling for user processes

In Alert Log:

Errors in file /u01/app/oracle/diag/rdbms/orclproddbone/orclprod/trace/psbcdb_vktm_3529.trc (incident=1200048) (PDBNAME=CDB$ROOT):
ORA-00800: soft external error, arguments: [Set Priority Failed], [VKTM], [Check traces and OS configuration], [Check Oracle document and MOS notes], []
Incident details in: /u01/app/oracle/diag/rdbms/orclproddbone/orclprod/incident/incdir_1200048/psbcdb_vktm_3529_i1200048.trc
2023-10-30T10:04:53.536165+06:00
Error attempting to elevate VKTM's priority: no further priority changes will be attempted for this process

In Trace File:

more /u01/app/oracle/diag/rdbms/orclproddbone/orclprod/trace/psbcdb_vktm_3529.trc

[oracle@dbupgradesrv ~]$ more /u01/app/oracle/diag/rdbms/orclproddbone/orclprod/trace/orclprod_vktm_3529.trc
Trace file /u01/app/oracle/diag/rdbms/dbupgrade/orclprod/trace/orclprod_vktm_3529.trc
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0
Build label: RDBMS_19.3.0.0.0DBRU_LINUX.X64_190417
ORACLE_HOME: /u01/app/oracle/product/19c/db_1
System name: Linux
Node name: Server
Release: 4.18.0-477.10.1.el8_8.x86_64
Version: #1 SMP Wed Apr 5 13:35:01 EDT 2023
Machine: x86_64
Instance name: orclprod
Redo thread mounted by this instance: 0 <none>
Oracle process number: 5
Unix process pid: 3529, image: oracle@Server (VKTM)

*** 2023-10-30T10:04:53.535009+06:00
*** CLIENT ID:() 2023-10-30T10:04:53.535027+06:00
*** SERVICE NAME:() 2023-10-30T10:04:53.535031+06:00
*** MODULE NAME:() 2023-10-30T10:04:53.535034+06:00
*** ACTION NAME:() 2023-10-30T10:04:53.535037+06:00
*** CLIENT DRIVER:() 2023-10-30T10:04:53.535042+06:00

2023-10-30T10:04:53.534502+06:00
Incident 1200048 created, dump file: /u01/app/oracle/diag/rdbms/orclproddbone/orclprod/incident/incdir_1200048/orclprod_vktm_3529_i1200048.trc
ORA-00800: soft external error, arguments: [Set Priority Failed], [VKTM], [Check traces and OS configuration], [Check Oracle document and MOS notes], []

*** SESSION ID:(2351.6983) 2023-10-30T10:04:53.570556+06:00
*** SERVICE NAME:() 2023-10-30T10:04:53.570568+06:00
*** CONTAINER ID:(1) 2023-10-30T10:04:53.570572+06:00

kstmmainvktm: failed in setting elevated priority
Verify: SETUID is set on ORADISM and restart the instance
highres_enabled
VKTM running at (100ms) precision
kstmrmtickcntkeeper: param _dbrm_quantum will not be effective
[Start] HighResTick = 209726095
kstmrmtickcnt = 0, ksudbrmseccnt[0] = 1698638693
[oracle@Server ~]$

Root Cause According to Red Hat: 

1. Short answer: The insights-client service unit sets a CPU quota of 30% on the system.slice
             /usr/lib/systemd/system/insights-client.service
2. Long answer:
On RHEL servers systemd is used to control services.
These services can ask for CPU quota.
In order to provide this CPU accounting, systemd creates the CGroup nodes for system.slice and user.slice, but it cannot set the value of the cpu.rt_runtime_us quantum.
This effectively disables realtime scheduling for all processes assigned to that slice
The system.slice is affected here.
The Oracle Grid software relies on the realtime scheduling.
There currently is no other effective way to limit CPU.

Diagnostic Steps According to Red Hat: 

1. The /var/log/messages log file contains:
Mar 31 02:53:36 example-grid-srvr1 kernel: Initializing cgroup subsys cpuset
Mar 31 02:53:36 example-grid-srvr1 kernel: Initializing cgroup subsys cpu
Mar 31 02:53:36 example-grid-srvr1 kernel: Initializing cgroup subsys cpuacct

2. Confirm changes on the system made by the insights-client
# find /etc/systemd/system.conf /etc/systemd/system /usr/lib/systemd -type f | xargs grep -e CPUAccounting -e CPUWeight -e StartupCPUWeight -e CPUShares -e StartupCPUShares -e CPUQuota |grep -v -e :# -e "^Binary file"
/usr/lib/systemd/system/insights-client.service:CPUQuota=30%

3. Removal of the insights-client package allows the Grid services to start

My Step:

  1. Remove insights-client package: [root@Server ~]# dnf remove insights-client
  2. Shut Down Database
  3. Reboot Operating System
  4. Open Alert Log: [oracle@Server ~]$ tail -f -n 22 /u01/app/oracle/diag/rdbms/orclproddbone/orclprod/trace/alert_psbcdb.log
  5. Start Database Gradually: nomunt -> mount -> open -> open pluggable database and frequently check Alert Log
  6. The error ORA-00800 is gone.

References: 

https://access.redhat.com/solutions/6991342
https://access.redhat.com/solutions/6020561
https://access.redhat.com/solutions/6991342
https://access.redhat.com/solutions/7004058



 
Labels:

Comments

Post a Comment

Popular posts from this blog

Upgrading Issue for RHEL 7 to 8 With Leapp

Overview The Leapp utility is a framework for updating and upgrading operating systems as well as applications. The operations of this utility consist of two phases 1. the preupgrade Phase – that chack the upgrade possibilities and 2. the actual upgrade phase – that map packages between previous and current versions of the software packages. Issue – 01: After running ‘ sudo leapp preupgrade ‘ sometimes you find the below issue in ‘ /var/log/leapp/leapp-report.txt ‘. Detail: Risk Factor: high (inhibitor) Title: Leapp detected loaded kernel drivers which have been removed in RHEL 8. Upgrade cannot proceed. Summary: Support for the following RHEL 7 device drivers has been removed in RHEL 8: – pata_acpi Key: f08a07da902958defa4f5c2699fae9ec2eb67c5b Remediation: 1. Disable detected kernel drivers in order to proceed with the upgrade process using the rmmod or modprobe -r . rmmod – Simple program to remove a module from the Linux Kernel modprobe – Add and remove modules from the Linux Ke...
Post a Comment
Read more

Upgrading Oracle Linux 6 to 7

Overview It is possible to upgrade an Oracle Linux 6 system to Oracle Linux 7.6 under the following conditions: The system meets the minimum installation requirements for Oracle Linux 7 as described in Chapter 1, System Requirements and Limits. The Oracle Linux 6 system has been completely updated from the ol6_x86_64_latest channel or ol6_latest repository. UEK R3 or UEK R4 has been installed on the system to be upgraded and is the default boot kernel. Upgrading from UEK R2 is not supported. Note that the system is upgraded to use the UEK R5 release provided with Oracle Linux 7.6. Upgrading is supported only for systems that are installed with the Minimal Install base environment. If additional packages are installed from an alternative repository or channel, upgrade might fail or the resulting upgrade might not function as expected. reference: https://docs.oracle.com/en/operating-systems/oracle-linux/7/relnotes7.6/ol7-install.html#ol7-upgrade-ol6 Verifying the system before Upgrade: #...
Post a Comment
Read more

Install Oracle Database 12c Release 2 On Red Hat 8

Overview According to Oracle, Oracle Database 12c is ‘the first database designed for the cloud’; the suffix ‘c’ stands for the cloud. There are many new features in this release such as multitenant architecture, pluggable database, in-memory, etc. The multitenant architecture is designed to simplify consolidation without requiring any changes to the applications. The rapid provisioning and portability capabilities are enhanced by the pluggable databases. Another new feature is in-memory makes it the first Oracle database to offer real-time analytics. This article describes the installation of Oracle Database 12c release 2 (12.2.0.1.0) 64-bit on Red Hat 8 64-bit. Lab Environment Server Machine Work Station or Client Machine OS Release Red Hat Enterprise Linux release 8.4 (Ootpa) Red Hat Enterprise Linux release 8.4 (Ootpa) Kernel 4.18.0-305.3.1.el8_4.x86_64 4.18.0-305.7.1.el8_4.x86_64 Release Oracle Database 12.1.0.2.0 Oracle SQL Developer Version 19.2.1.247 IP Address 192.168.201.116 ...
Post a Comment
Read more