| [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] |
Original question:
> I would like to hear from the people on this list how you handle
> change control on your linux/unix servers.
>
> Maybe if I ask a few questions it might give some direction to the
> answers:
>
> 1. What constitutes a change that has to be controlled in such a way
> that you have to get permission to do it?
> 2. How do you handle security updates? Is the administrator allowed
> to do a daily "apt-get update" and "apt-get dist-upgrade" on
> production servers?
> 3. How do you apply for permission to change something on the server?
> Using email, paper forms, webforms?
Thank you for the following people who replied (one requested to
remain anonymous): Sean Lamb, Edward Macnaghten, Brett Geer, Kelly
McDonald and Angel Rivera.
The answers helped me a lot and contained a wide variety of
approaches. Here are some quotes, remarks and summaries:
* Install security or critical update to software as soon as possible
and schedule other updates to happen at the same time.
* The administrator for the server is responsible for security
upgrades and normal administrative tasks. Redhat up2date is used for
upgrades. Changes in strategy or creation of new services are done
after consulting with a higher level of management.
* All changes required must be motivated to the section
manager/supervisor who then authorises it and when it's completed
and tested then the system files (paperwork) is updated. writting
procedures regarding how change control is to be done.
* Security updates should be done to a test system first, then QA
systems and if successfull then the production systems.
* Some use paperwork for the adminstration, others meetings or ticket
software.
* The following is a more complete report back about an example I
thought might be particularly valuable in our situation There was
also a second one with the same ide of a regular Change Control
meeting):
In one instance there is a change management council where they have
time blocked off weekly on Monday nights in order to perform
whatever work is necessary. These Preventative Maintenance periods
(PM's) typically last 4-5 hours.
All software releases slated to go into production are released
during that PM window. All changes have to be submitted
electronically before or in person at a 1PM Friday change management
meeting. At the PM meeting are members of our QA group, our
software configuration management group, our frontline support
staff, and technical leads from production. All items are assessed
for impact to the productions, and work is coordinated such that
people don't screw each other up. The needs of the productions are
certainly taken into consideration during that meeting, and work may
get moved to a different week to reduce risk to the productions.
On Monday afternoon, a note is sent out to everyone reminding them
that they'll be logged out at 9PM, and stating for the record the
intentions of the evening and the assessed impact.
There is a PM kick-off script that shuts down service monitoring and
halts batch processing systems, and notifies the sysadmins that PM
has started. Sysadmins perform their work roughly to the plans that
were reached via consensus (no written plans needed, presently,
unless a particular change deserves it). Sysadmins are responsible
for testing what they believe they impacted, and write up a PM
report before they leave. The last person out runs a close-of-PM
script to renable service monitoring and batch processing systems.
At 7AM, the first day shift of front-line support folk come in and
go through a series of manual checks of things like graphical apps
to make sure everything passes.
Software releases, including security errata, are discussed in
various meetings as appropriate before the Friday PM meeting. We
never pull directly from a 3rd-party source as is the case with
apt-get, and all packages and software is tested at least to some
sane extent by sysadmins, software developers, QA folks, and
sometimes even production folk. The use of apt-get to update to a
known-tested local repository would be acceptable, though. We use
rpm locally, so we have an extensive set of scripts to pull in
security errata to testing areas, and then we generate update
scripts that get ran on all of our hosts.
* One good rule of thumb is to say, "Is there any risk that my users
will notice the slightest hiccup?" If the answer is 'yes', then you
should get permission.
* Schedule changes during a standard maintenance window. Getting
permission is something else.
* Changes to the development environment should be as simple as, "What
will this impact?" ... When in doubt, consult your peers, bounce it
off your manager.
* The production environment should be fairly unchanging. All changes
to that environment should be approved by anyone/everyone who has
ownership/responsibility for the applications.
Regards.
Johann
--
Johann Spies Telefoon: 021-808 4036
Informasietegnologie, Universiteit van Stellenbosch
"Surely he took up our infirmities and carried our
sorrows; yet we considered him stricken by God,
smitten by him, and afflicted. But he was pierced for
our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and
by his wounds we are healed." Isaiah 53:4,5
_______________________________________________
LinuxManagers mailing list - http://www.linuxmanagers.org
submissions: LinuxManagers@linuxmanagers.org
subscribe/unsubscribe: http://www.linuxmanagers.org/mailman/listinfo/linuxmanagers
[Home] [Kernel List] [Linux SCSI] [Video 4 Linux] [Linux Admin] [Yosemite News] [Motherboards]