Drew Community  

Go Back   Drew Community > General Forums > Technology Discussion
uLogin ID  
Password
FAQ Members List Calendar Search Today's Posts Mark Forums Read


Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 02-08-2009, 12:59 AM
E. Axel Larsson's Avatar
E. Axel Larsson E. Axel Larsson is offline
Moderator
 
Join Date: Jun 2005
Location: Madison, NJ
Posts: 303
Default Web service problem

We had a problem this evening with one of our three iChain servers. The iChain servers sit in front of all of our web-based services and provide single-sign-on capability, allowing you to log into all of Drew's services such as Webmail, CampusWeb, Moodle, this site, etc. once with your Drew uLogin ID and password.

It looks like one of the three machines started to develop a memory issue around 8:00 pm which continued until it was rebooted after midnight at around 12:30 am. During that time, that machine was not authenticating new users. New users trying to log in would receive a 500 Internal Server Error. Since this was only affecting one of the three machines and they are load-balanced the problem would not have impacted all users, and most likely closing browser and logging in again would send your session to a different machine which was working.

Since none of the services on the iChain machine actually crashed the failure was not detected by our monitoring system. The affected machine was restarted shortly after receiving a user report of the problem.

We apoligize for the outage and I am currently looking into options to enable our monitoring system to detect this type of problem in the future.
__________________
E. Axel Larsson
Systems Architect and Director of the Enterprise Technology Center
Reply With Quote
  #2  
Old 02-09-2009, 01:36 PM
E. Axel Larsson's Avatar
E. Axel Larsson E. Axel Larsson is offline
Moderator
 
Join Date: Jun 2005
Location: Madison, NJ
Posts: 303
Default

Quote:
Originally Posted by E. Axel Larsson
We had a problem this evening with one of our three iChain servers. The iChain servers sit in front of all of our web-based services and provide single-sign-on capability, allowing you to log into all of Drew's services such as Webmail, CampusWeb, Moodle, this site, etc. once with your Drew uLogin ID and password.

It looks like one of the three machines started to develop a memory issue around 8:00 pm which continued until it was rebooted after midnight at around 12:30 am. During that time, that machine was not authenticating new users. New users trying to log in would receive a 500 Internal Server Error. Since this was only affecting one of the three machines and they are load-balanced the problem would not have impacted all users, and most likely closing browser and logging in again would send your session to a different machine which was working.

Since none of the services on the iChain machine actually crashed the failure was not detected by our monitoring system. The affected machine was restarted shortly after receiving a user report of the problem.

We apoligize for the outage and I am currently looking into options to enable our monitoring system to detect this type of problem in the future.
We have now implemented some additional service monitoring so that we will get an alert notification if this specific issue occurs again.
__________________
E. Axel Larsson
Systems Architect and Director of the Enterprise Technology Center
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -4. The time now is 09:37 AM.


Powered by vBulletin® Version 3.5.7
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.

Drew University is not responsible for the content of posts made on this site. All posts and comments reflect the opinion of the author.