Ray Butterworth — 2013 August 21
At MFCF's June 17 Management meeting, Wayne asked me to investigate the effects of getting rid of Xhier.
An initial reporthttps://rbutterworth.nfshost.com/Reports/unxhier/unxhier-1/ outlined the services provided by Xhier; indicated that while some of those services could be better provided by other products some would need considerable work to be continued without Xhier; outlined the direction currently being followed by IST and CSCF; and suggested removing the need for some or all of those other services as a direction to proceed.
Wayne said that yes, the Xhier software will disappear, and eliminating unessential services is our direction (and not only as far as Xhier is concerned).
The need for this service is created by our policy of making multiple architectures appear as similar as possible to our users.
The need for this service is created by our policy of allowing people to use a machine as if we hadn't made any changes to it. (It also reduces the work required to install system patches and perform OS upgrades.)
The need for this service is created by our policy of allowing users to have access to different versions of the same package on individual machines.
The need for this service is created by our policy of providing the same software on multiple architectures.
The need for this service is created by our policy of making administration of multiple software packages as easy to learn and manage as possible.
The need for this service is created by our policy of providing a hierarchical administration structure.
The need for this service is created by our policy of providing software on multiple hosts of the same architecture.
The need for this service is not policy, but a matter of staff wanting to do things in the most productive way possible. Maintaining ten machines should not require 10 times as much work as maintaining one. Similarly, when there was a large enough number of people (critical mass) working with Xhier, all campus groups involved with it benefited from the work done by other groups.
Most of, or perhaps all, the above services are essential in order to fulfil past and present MFCF policies. They are not necessarily essential in order to fulfil our mandate, and in many cases not essential in meeting the current needs of our clients. Even in cases where those needs exist, they might be provided by other means.
Xhier was developed to meet a real need. At one time we had nearly 20 different architectures in our student labs and research environment. Users would have become very confused had they met with significantly different behaviour depending upon which terminal they happened to have sat down to use.
We now support only 4 major architectures (Mac, Windows, Solaris, Linux), and only two of them are similar enough that anyone might be confused by their minor differences. (This raises the issue of whether there is still any need to support both Solaris and Linux.)
Virtual machines are effectively free, easy to generate, and have a more pristine environment than what we are currently providing.
Again, without the policy of providing one large seemingly homogeneous environment, the need for providing multiple versions of software on the same machine goes away. Where there is a need for a specific old version, a virtual machine for instance could be set up specifically for that purpose.
(On the other hand, a more heterogeneous environment will
conflict with our more recent policy of presenting a simplistic
view to our clients (e.g. linux.math rather
than having to know about specific cpu### names).)
And again, if we do not have a policy of providing a uniform environment, the need for building the same software on all architectures becomes much less. The amount of software that we require in multiple places might actually be quite small.
This is still a good goal, but given the large number of non-local packages that we now provide and support, even achieving perfection among the xhiered packages would not have a great overall effect.
Having IQC able to set their own software policies and configuration, within those set by MFCF, within those set by IST, has proven to be useful. But more and more, groups are setting up smaller self-contained computing that doesn't need the generic environment we provide.
Centralized configuration for the machines MFCF maintains is still a good and necessary goal, but without a policy that requires hierarchical administration, there are tools other than Xhier that provide this service.
While in some ways better than, and in some ways not as good as Xhier, these same centralized configuration tools (e.g. Puppet) can provide, or greatly help with, the distribution of software.
Moving to a more heterogeneous environment will make the automation and scaling of some tasks more difficult.
Eventually, it's possible that we might be in a position to develop modules for open some source packages that might benefit a community much larger than the current Xhier world.
If MFCF were to make large changes to many of our long-time policies regarding what software services we provide, as described above, most of the need for Xhier would go away.
We would support only one major version each of Windows, MacOS, and Linux. The first two are outside the scope of this report, but all Linux machines would be administered from one central place by one single authority (presumably MFCF).
We don't currently know what software our clients need that is dependent on Solaris and not provided by Linux. Once we have eliminated mail services, in particular IMAP, we should be able to get a much better view of how our student and research regions are actually being used by our clients. The usage statistics for October or November should give us a good sample to work with.
We do know that a significant number of programs and software packages currently relying on Xhier for compilation, configuration, distribution, etc. are used by MFCF itself.
Some of those are part of our centralized administration and accounting systems, and so really need to exist on only one host. But others are required on all our client hosts. In either case, the current source base will not build or install without the tools and environment provided by Xhier.
We might be able to reduce the amount of such software by eliminating or reducing the need for the services that they provide. This will require considerable study and further policy changes. Even then, for what remains as essential we will need to make large changes to our source and our procedures.
A survey of MFCF staff indicates that (at least) the following software packages, some currently highly dependent on Xhier, are needed.
| Package | Purpose | Future |
|---|---|---|
| accounts_client | Manages user resources. | ??? |
| accounts-master | Enables us to know safe-to-remove resource allocations. | Being partially reviewed/replaced by Wayne et al. |
| batch | Serializes execution of background tasks. | ???. |
| gnomeconfig | Simple configuration for the Gnome window manager. | ???. |
| gnu | Provides modern versions to cover old Solaris software. | Not needed if Solaris goes away. |
| hostselect | Select the 'best' host based on configured criteria. | ??? |
| linux-extras | OS-specific package for Linux. | ???. |
| lpr and friends | Print submission, access control, accounting, and printing. | ??? |
| mfcf-accounting | Collects usage data. | Being reviewed/replaced by Wayne et al. |
| mfcf-accounting-master | Analyzes data, produces bills and reports. | Being reviewed/replaced by Wayne et al. |
| mathcpuserver | Configuration overseer package for CPU server machines in the Math region. | ???. |
| mysql | Database and interface. | Available on Linux. |
| nagios-* | System monitoring. | ??? |
| nmap | Network exploration tool and security/port scanner. | Available on Linux. |
| ntp-config | A "universal" xntp3 configuration package. | ??? |
| perl | For various Perl scripts. | Available on Linux. |
| printcap | Maintains and distributes printcap data. | ??? |
| rcs | Revision control system. | ??? |
| rt-math | Interface to MFCF and CSCF request tracking. | ??? |
| security | Controls root access, reports unusual files, etc. | Not needed if Solaris goes away. |
| sunos5 | SunOS5.* OS specific package. | ??? |
| setpw | Updating and distributing passwd and group files. | ??? |
| titrax | For tracking time usage. | ??? |
| uid-registry | Central database server for UID and GID values. | ??? |
| wwwdata_* | www.math and other web data and configuration. | ??? |
| x11-mfcfenv | Provides basic TWM environment for X11 with connection to dtlogin in Solaris 10 CDE. | ??? |
In addition, a number of specific programs were requested.
| Program | Purpose | Future |
|---|---|---|
| acroread | For displaying PDF documents. | Is available on Linux. |
| align | For quick formatting of data. | ??? |
| decomment | For extracting real data from config files etc. | ??? |
| dump_inventory | Formatted dump of selected inventory fields and records. | ??? |
| ffind | Cached database of filesystem. | Available on Linux as locate. |
| gpdf, gsview | X11 graphic display of PDF and PostScript. | Available on Linux as evince. |
| gvim | Graphical version of VIM | Is available on Linux. |
| host | DNS lookup. | Is available on Linux. |
| hostin | Information about hosts and relative attributes. | ??? |
| hostinfo | DNS information and inventory data. | ??? |
| lc | Nice format of directory contents. | ??? |
| logsum | Summarizes system mail sent to root. | ??? |
| lsof | Lists open files (debugging and security). | Is available on Linux. |
| prsh | Runs command in parallel on multiple hosts. | ??? |
| ssh, ping, traceroute, netstat, ntpdate | Various handy programs. | All available on Linux. |
| truss | Debugging utility. | Linux has strace and ltrace. |
| vi, cron, sort | Various handy programs. | All available on Linux. |
| wget | Quick download of web data. | Available on Linux. |
Some people also suggested packages that they thought our clients might need.
| Program | Purpose | Future |
|---|---|---|
| R, maple, matlab, sage, sas | Required for teaching. Packaged using Xhier. | ??? |
| buildbot, gsl, mpitch, splus, stata, twisted, xanim, zope | Required for teaching. Packaged using Xhier. | ??? |
Responses have been received from Chris, Jim, Lori, Ray, and Robyn.
For completeness and to eliminate personal bias, the tables above include all submissions, even those that might have immediately been omitted.
(More investigation is needed into why some of these packages and programs are needed.)
Many questions remain unanswered. Some are technical, requiring more investigation, while others are more a matter of policy and direction.
Once we have a better understanding of where we hope to be in a year or two, we can start working toward that goal.
A good place to start would be to construct a small environment of say one central server and a separate region of two hosts; to build and install our essential software on them; and to ensure that configuration, resource management, accounting, etc. all work as required.
Once that is working to our satisfaction, we can add more hosts into this new world, and phase out our current environment.