PeopleTools 8.51 & Exchange 2010 revisited

We went into production on the Exchange 2010 server last Friday.  The Exchange server admins were able to set the TarpitInterval based on where the email was originating.


Also we changed the code that sends hundreds of emails in TAM to a batched process.  

Here are the exchange commands we used and the settings changed.

Set-ReceiveConnector –Identity servername\relayconnectorname –ConnectionTimeout  00:20:00

Set-ReceiveConnector –Identity servername\relayconnectorname -TarpitInterval  00:00:00

Set-ReceiveConnector –Identity servername\relayconnectorname –MaxAcknowledgementDelay 00:00:00

External Applicants and Browsers

Right after go-live from Tools 8.49 to 8.51 we started getting calls from external applicants with many different issues.  Most of these callers were using unsupported browsers.  We increased the visibility of our message that told applicants what browsers are supported.  But that didn't help for a couple of reasons.  Number one --some of our applicants get to our system through links from job search sites.  Our message about browsers was on our company careers web page.  The second reason was that people don't pay any attention to those messages.

This issue frustrated the recruiters because the number of applications coming in was down.  And this issue frustrated developers because we couldn't track down a particular page or component that caused it.  It was an intermittent issue that we couldn't reproduce.  The application server logs didn't offer any helpful information about the errors.

After finding this post on AJAX issues, we turned off AJAX for our External Applicant web profile.  Follow the link to get detailed instructions.  The basic idea: on Web Profile Configuration / Custom Properties tab, create a boolean property "AJAX" with a value of "false".  It changes the user experience, but that change was minimal in the application process.  I think there is one search that goes to a search page now, while with AJAX it used the modal search window.



Update:
See the later post on the 501/AJAX error here.

PeopleTools 8.51 & Exchange 2010: Part 2, The Options

We opened a case with Oracle about the slowdown.  PeopleSoft Development said they had seen this five second per email issue before, but weren't able to get a customer to test it to provide "concrete evidence".  We sent them SMTP trace logs from the application servers and we sent them logs from the Exchange server.  They wanted us to test with Javamail 1.4.3, rather than the Javamail 1.3 that was supplied with PT 8.51.  Evidently PeopleSoft development was using 1.4.3.  The newer version of Javamail will not to send the NOOP command, but will send RSET command which might solve the problem.  Javamail 1.3 does not have this feature.  They wanted us to try it out to see if it worked, but they couldn't recommend that solution since it hadn't been fully tested.  It would be standard in some future version of PeopleTools.

The option that the PeopleSoft developers recommended was to have us to try setting the TarpitInterval to 0.  From the Microsoft Technet link, "The TarpitInterval parameter specifies the period of time to delay an SMTP response to a remote server that Exchange determines may be abusing the connection. Authenticated connections are never delayed in this manner. The default value is 5 seconds. ...When you set the value to 00:00:00, you disable the tarpitting interval."

The Exchange server admins were loath to disable the new security feature that they had just gained with their upgrade.  They didn't want to change a global setting on their servers for just one application.  They were not able to give us our own Exchange server.  I asked about setting the PeopleSoft application servers as "authenticated connections", but was told that it was too much work to pass credentials.

The other option was to rewrite the code in TAM that sent the emails to all the rejected applicants.  It is a custom process.  The delivered process sent snail-mail to the applicants, and as a cost saving measure we changed it to send email to all the applicants that had email addresses entered.  The easy solution would be to cut the PeopleCode out of the process and paste it into an Application Engine process.  Using the same code would be easy, but there would still be a five second per email delay later.  The other option would be to rewrite it in SQR.  The SQR would use the Unix sendmail, and so would run faster.  Either of those options would handle the emails in a batched mode.  The user would no longer need to wait for the email to be sent before moving on to the next screen.  While it may seem a bad practice to have the user wait while we send an email, you need to remember that with the previous Exchange server we could sent 100 emails in one second.  The downside for these solutions is that it doesn't address the five second per email delay for the email sent from the application server.  It will affect other parts of the system later.  We want to deploy more AWE (Workflow) processes either as part of or after the upgrade.