ORA-27300: OS system dependent operation:invalid_egid failed with status: 1

My Oracle DBA called me this evening and complained his production database is down & not accessible.

Obviously I asked him for the error log and he provided me this.

ASMB started with pid=32, OS id=11239
ORA-00210: cannot open the specified control file
ORA-00202: control file: '+ARCH_1/gsicp/controlfile/current.263.700590259'
ORA-17503: ksfdopn:2 Failed to open file +ARCH_1/gsicp/controlfile/current.263.700590259
ORA-15001: diskgroup "ARCH_1" does not exist or is not mounted
ORA-15055: unable to connect to ASM instance
ORA-27140: attach to post/wait facility failed
ORA-27300: OS system dependent operation:invalid_egid failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: skgpwinit6
ORA-27303: additional information: startup egid = 301 (oinstall), current egid = 305 (asmadmin)
ORA-00210: cannot open the specified control file
ORA-00202: control file: '+DATA_1/gsicp/controlfile/current.357.700352197'
ORA-17503: ksfdopn:2 Failed to open file +DATA_1/gsicp/controlfile/current.357.700352197
ORA-15001: diskgroup "DATA_1" does not exist or is not mounted
ORA-15055: unable to connect to ASM instance
ORA-27140: attach to post/wait facility failed
ORA-27300: OS system dependent operation:invalid_egid failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: skgpwinit6
ORA-27303: additional information: startup egid = 301 (oinstall), current egid = 305 (asmadmin)
ORA-205 signalled during: ALTER DATABASE MOUNT...
Mon Mar 11 07:48:21 2013
Shutting down instance (abort)
License high water mark = 2
USER (ospid: 13798): terminating the instance
Instance terminated by USER, pid = 13798

I found this issue is quite common when they perform Oracle upgrade as they tend to mess up the file permission.

Solution for this problem is quite simple, go view this file

-rwxr-x--x 1 oracle asmadmin 232473728 Mar 11 03:59 /u01/app/oracle/product/11.2.0.3/db_1/bin/oracle

Change file group ownership back to oinstall and restart the database/ASM instance & you’re good to go.

-rwxr-x--x 1 oracle oinstall 232473728 Mar 11 03:59 /u01/app/oracle/product/11.2.0.3/db_1/bin/oracle

Please take note your environment may be different from mine, just pay attention to the error log to determine what is the correct group for your oracle grid binary file.