When I first started moving GroupWise systems to SLES 12, I had a horrible time just configuring GroupWise to capture a core in the event of a crash. It took a long time to figure it out since it’s not as automatic as on SLES 11. SLES 15 would also have the same nuances as SLES 12. I was finally able to compile a list of needed items to either try to prevent the crash or create a good core once it does. Also I was told by GroupWise support that these cores provided better  info that the cores produced by default on SLES 11. In a situation where I can expect a core, I do this ahead of time as a necessary measure. I don’t wait to see if it cores and then try to do this, I just do it.

Note: Apparmor tends to interfere with cores so ensure it’s turned off or disabled. I had a customer reboot a server last week, then GroupWise crashed, but I had not disabled AppArmor, only unloaded it. So when the server restarted, it loaded again and I failed to get a core at all.

/etc/sysctl.conf

Add the following lines to /etc/sysctl.conf:

kernel.core_uses_pid = 1
kernel.core_pattern=/opt/cores/core.%p
kernel.suid_dumpable=2

Then issue the following command :

sysctl -p /etc/sysctl.conf -w

/etc/sysconfig/grpwise

Add the following line to /etc/sysconfig/grpwise:

GROUPWISE_DEBUG_OPTION=”on,fence”

Note #1:  It’s /etc/sysconfig/grpwise not /etc/init.d/grpwise.
Note #2: This is case sensitive and should be entered exactly as shown

/etc/init.d/grpwise

Check the /etc/init.d/grpwise file, you should see the following string (It’s usually there):

“$GROUPWISE_DEBUG_OPTION”

Add the following statement in the /etc/init.d/grpwise script. (put it under other export statement)

export GW_MEMTST=on,fence

noelision

The noelision option is not really about creating a core, it’s more about preventing a crash in the first place. There are some applications that are susceptible to an Intel CPU hardware bug.  GroupWise seems to be one of those applications. Reference this SUSE TID: https://www.suse.com/support/kb/doc/?id=7022289

Create a file called:

/etc/ld.so.conf.d/noelision.conf

Add the following line to the /etc/ld.so.conf.d/noelision.conf file you just created:

/lib64/noelision   (This text should be in the file)

Save the file and run the following command from a terminal prompt:

/sbin/ldconfig   (This is a run command)

SLES12/SLES15 Specific Core Process

The following steps should be taken to prepare for capturing a core dump on SLES 12/ SLES 15

  • Disable the limit for the maximum size of a core dump file
  • Configure a fixed location for storing core dumps
  • Disable AppArmor
  • Enable core dumps for setuid and setgid processes

The Process to accomplish is as follows:

Run the command:

ulimit -c unlimited (I find that this is generally already set.)

Run the command:

install -m 1777 -d /opt/cores  (This is my preferred path for the Core location, but you can change this)

Run the command:

echo “/opt/cores/core.%e.%p”> /proc/sys/kernel/core_pattern   (Adjust the path if necessary)

Run the command:

rcapparmor stop

Disable AppArmor in Yast:

Yast –> System –> Services Manager –> apparmor –> Disable (Scroll down the list, it’s not in alphabetical order, I find that it’s generally toward the bottom)

What to do if/when GroupWise crashes

If GroupWise crashes, it should now create a crash dump file (core) in the path specified above. The core file can be analyzed by GroupWise support to attempt to determine the root cause of the crash. You should do the following:

  • Restart the Post Office, it should just start right back up.
  • Open an SR (Service Request) with Micro Focus, Reference a Crashed POA that caused a critical outage. Priority High or Critical.
  • Create a tar.gz file of the core file and name it the same as your SR number. For example SR12345678910tar.gz. (The core file can be large so compressing it helps manage the size and upload time)
  • Upload to ftp://ftp.novell.com/incoming and notify the engineer of the file name.  Request that they have the core read.
  • Wait for them to tell you to upgrade to a newer FTF to see if it helps (I say this jokingly but we’ve all been there).