Showing posts with label history. Show all posts
Showing posts with label history. Show all posts

Sunday, November 9, 2008

Unix And Linux Humor - Cult Satire

Hey There,

For this Sunday's humor section, I found a cool little piece of future-nostalgia on a site called Hillside Systems. There's a very interesting article on their site (recently updated - Something from 2008! Almost unheard of ;) regarding The Unix Cult that's pretty enjoyable, and funny, reading if you like your satire straight up - no emoticons ;) <-- Like that winking thing I do every 5 seconds...

Enjoy, as we travel, now, into the future, to reflect upon (and attempt to understand) the past, which is probably happening as I type :)

Happy Sunday :)





The UNIX cult









Peter Collinson (local name)

AS/103/108/121/110 (Galactic designation)


Galactic Cult Investigation Team,

Canterbury, Kent

(Small Island off Continent 3, Sol 3)


Abstract

Notes from some recent archeological findings on the birth of the
UNIX cult on Sol 3 are presented.
Recently discovered electronic records have shed considerable light
on the beginnings of the cult.
A sketchy history of the cult is attempted.

Background

The UNIX
cult is widespread across the Galaxy now and the
surprise discovery of some ancient files in the archives of
Intergalactic Brain Machines on Sol 3 triggered the dispatch of
an inter-disciplinary investigation team.
The files are extremely
extensive, occupying all of a small island off the coast of
Continent 3.
It transpires that the island was
taken over by Intergal in the aftermath of
the Corporate Wars which plagued Sol 3 some centuries after the
birth of the cult.

The team were asked to find out the original meaning of some of
the incantations used in UNIX religious practice
and also to shed some light on what it all meant at the start.

We should take this opportunity to use the ancient prayer:

UNIX is a trademark of AT&T in the USA and other countries.

Earlier versions of this prayer do seem to exist, it is unclear why the
form of words altered.
`AT&T'
was the Corporation where the Creators of the cult worshiped.
The Corporation
totally disappeared in the wars and many of its original records were
either destroyed or altered by the victor in an attempt to `re-write'
history.
The placement of the country USA
on the four continents has been lost.

The Gurus

There seems to be still no trace of the original Creators of the UNIX
cult, so we start our examination of the records with a group calling
themselves the Gurus .
The etymology of this word is not quite clear
but it does have associations with religious teaching and the High Priests of
UNIX are given that title today.

Extract from electronic phone tap of someone nicknamed
`dsw', believed to be Daniel Stuart Wilson:


``Of course, we were all isolated in the Version 6 days.
When we changed things in the kernel we were on our own.
There was no-one to phone to scream for help, in many cases
there was no-one on the same site who you could discuss the problems
with.
We just had to get down and read that C.
Sorting things out yourself was painful but you benefited in the long
run.
You became a better software engineer.
After a bit you became a Guru and could lead others.
This is largely bluffing - given a new situation and a knowledge of
UNIX you could extrapolate the problems easily and seem to be very clever.
It didn't actually matter what you had or had not done, it mattered that you
appeared
to have done a lot and could talk confidently about
RK05's,
PDP-11/45's,
typing chdir rather than cd and a myriad other things
which showed that you were a Guru.''

This extract shows clearly that the cult grew in small
pockets across the globe.
In each centre, a few individuals were given the task
of installing the paraphernalia of the UNIX
cult and of converting others to its use.
In the beginning,
it seems that these individuals had little or no contact with each other;
curiously, this appears to have strengthened their ability rather than weakening it.
Other extracts from the archive indicate that
the early practices of the cult were small and simple making it easier for
one person to grasp their full meaning.
As the cult grew, the practices became more complicated and the understanding
harder.

The term
software engineer refers to a person whose task it was to feed instructions or
programs
into the primitive versions of the Overlords which were extant at the time.
Judging from the emphasis made in the records about the
need to generate `correct' programs, this was obviously an
artistic task requiring considerable expertise.
The reference to `C' implies that there was some official form of speech
or perhaps a special religious language which was used to convey instructions.

It is not clear whether the
RK05 and the PDP-11/45
were the names of the Overlords or some ancillary equipment associated
with them.
However,
the word cd
(Pronounced `see-dee')
meaning `to move from place to place'
is still used in some arborial societies on the planet US/115/110/105/120.

The User Group

In current UNIX
religious practice, the term
User Group
is used to refer to the congregations at the Hologram Services.
It seems that after a while
the early practitioners of the cult
began to have meetings (where people actually appeared together
in the same room).
Why these were called `User Group' meetings is unclear, since
the early meetings were attended by Gurus and not by Users.
At the time, a User seems to have been a derogatory term for
the un-initiates.
However, the head Guru at a site was given the title `Super-User';
perhaps this was to hide the evangelical nature of the Guru's task.

The reason for the early meetings was mostly to allow
Gurus to inform each other
how best to perform religious conversion and how to get the most converts
the fastest by improving the Service to Users.

As the word spread,
the meetings grew in size, expense and quality of surroundings.
They proved to be an exceptionally good way of upgrading
low level Novices into higher level Gurus.
This is a reflection on the early religious books which were
aimed at Gurus and were often way above the head of the Novices.
Novices were encouraged to attend the meetings to gain
by word of mouth what they could not gain from the literature.
At these later meetings, the nuts and bolts of UNIX
practice were still discussed because many of the Novices
were either acting
in a support role to Gurus or planning to become evangelists themselves.
After a while, the Gurus got bored with discussing
inodes (The meaning of this is totally lost).
and other topics began to creep in.

The meetings gradually altered in character, with the Gurus attending
because they wished to see the other Gurus; and more and more Novices
attending in the vain hope that they would learn something.
The cult had spread so far by now that it was profitable for
Corporations to become involved in the selling of UNIX
paraphernalia and religious goods.
Initially, these Products were advertised widely at the meetings.
However, the Corporations often sent attendees with prepared scripts
who were sometimes not even Users
and who had little or no knowledge of intricacies of UNIX
practice.
The Gurus and Novices were dismayed.

On their side, Corporations began to see that the word which
they were spreading was falling on somewhat stony ground.
This gave rise to the Great Split with a rival organisation being
set up primarily aimed at selling and the original User Group
concerning itself more with the cerebral activities of the cult.
In the end, this was a good thing because the Gurus were able
to start deriving benefit from the meetings again since
there were now spare slots for educational talks.
In fact,
the rival organisation was wiped out in the Corporate wars
because it had allied itself strongly with AT&T.

The User Group, then, provided some important functions.
It supplied a forum for discussion of the practices of the cult.
It provided a meeting place for the widespread Gurus who initially
met to discuss their work but as time passed they went just to meet
each other.
It spread the word to Novices; and
as the cult grew, it provided a place where new ideas on the
direction which things should take could be discussed.

Another more hygienic method of worldwide communication
grew out of the cult with the formation of the Network.
This is discussed in the next section.

The Communicators

When the cult had grown to worldwide proportions, we begin to see
the emergence of a global communication system - the `Network'
or `Net'.
The activities on the Network can be deduced from the
electronic archives but the material is so vast that
scanning it for relevant information is proving difficult.

The majority of traffic on the network seems to have been
communication between Overlords describing various error conditions.
Here is a sample:




Various repeated unintelligible lines
From: MAILER-DAEMON (Mail Delivery Subsystem)
To: <uucp@a4los.uucp>
Subject: Returned mail: Service unavailable
----- Transcript of session follows -----
>>> DATA
<<< 554 sendall: too many hops (30 max)
554 <megd> ... Service unavailable: Bad file number
----- Unsent message follows -----
More of the same



There are very many other examples of the same type of message.
However, these messages do place things in context - we now know
that the communication system was called Mail and from this we
infer that the mechanism was intended to permit communication between
people.
In amongst all the Overlord messages we do find some files
which appear to emanate from one person and be addressed to another.
A high proportion of the sampled files were intended to probe the
capabilities of the Network, often provoking an Overlord error message.
We know this because the
destination address is the same as the source; in some cases the
subject is `Just testing' or something akin to that.
This technique was perhaps used to investigate the ability
of the Network to transfer files.
We are left with a much smaller
number of what might be termed `useful' messages.
In many cases, these consist of personal trivia showing that
the Network was supplying the useful social function of allowing
people to make and maintain contact.

Other messages consist of hieroglyphics containing many
braces `{' and brackets `('.
This type of message shows some form of regularity in structure but
syntax analysis is hampered by the presence of so many exceptions.
It is possible that these messages contain portions of religious
ceremonial or it has been suggested that this is the
ritual language `C' and the exceptions are what were called `bugs'.
This last suggestion (by a post graduate student on the team) has
been greeted with a certain amount of derision.

Mail messages have a recognisable format and can be distinguished from
another type of message which occurs in much greater volume.
It appears that these messages are part of a system called `The News'
broadcast to many sites.
The News permitted individuals to send a single message to many
people across the world.
A sample of the contents imply that the News took over as
the main method of spreading the UNIX
word when the User Group meetings ceased to be totally useful
in this role.

Judging from the beginning of many News files,
the News was split into many separate subject headings.
Over large periods of time, we see the subjects come and go with
abrupt changes in title occurring from time to time.
The subjects do not seem to be confined to discussion of topics
with direct relevance to the cult.
There are many headings which just carry `talk' on various subjects.
An analysis of the topics is being prepared as a background
document since they fall outside the remit of the investigation.

It might be thought that the News would provide an excellent vehicle
for the Super Gurus who must have existed by this time - but
far from it - very few messages contain what might be termed
confident information.
Most seem to carry opinion which is contradicted in later messages.
The evidence here is that most authors were Novices.
The Gurus had either lost interest in spreading the word, or were simply
too busy to wade through what must have been daily oceans of verbiage.
It is also possible that Gurus just used the Mail to communicate, perhaps
not wishing to impose their definitive opinions on the discussion
with the thought that discussion is a healthy academic tool.

The Network spread slowly over most of Sol 3 with some areas
being exempt because either they resisted conversion to the UNIX
cult or were deliberately omitted for ideological reasons.
A single News message from `kremvax', an otherwise silent site,
seems to have been met with a storm of protest.
This shows that the Network crossed political boundaries and proves
the contention that Sol 3 was split into separate economic entities
predating the rise of the Corporations.

The Vendors

The Corporations who were involved in the propagation of UNIX
paraphernalia and religious goods were known as
Vendors.
The Vendors had a different view of the world from the Gurus, and
this difference lead to many schisms in the cult.
The Vendors and Gurus tried to maintain a separate identity at all times.
For example, when the Vendors attended the User Group meetings
they provocatively wore different religious vestment from the Gurus.
They did this to make it easy for other Vendors to identify them
and to allow their easy differentiation from the Gurus and Novices.

At the centre of the clash of opinion was a fundamental difference
in the perception of `the User'.
The Vendors were always complaining that UNIX
religious practices were not `User friendly'.
By this, they meant that the act of worship was hard to learn, and
the rituals were cryptic.
They wished their Users to only deal in simple concepts and never
to learn more than the basics of the rituals.
The Gurus objected to this view because they believed that the learning
of cryptic rituals allowed the worship to proceed faster and
that limited exposure to only the simple concepts restricted the
Users in a way which was not desirable.
The Vendors believed that the Users were fundamentally stupid and
without any hope of redemption; and the Gurus believed that the
Users were fundamentally stupid but might be saved given the correct
tuition.

The early Vendors were peopled by Gurus who had often to undergo the
necessary clothing transformation to demonstrate that they had
switched camps.
These Vendors were responsible for the spreading of the cult
to a much wider and naive User population.
Some of the Vendors tried to set up their own breakaway cults to avoid
the central control imposed by AT&T,
the Corporation where the Creators worshiped.
These attempts failed.
Other Vendors created Sects which specialised in worshiping the
many minor Overlords which had appeared about this time.
These Sects were partially successful and converted Users
to their way of thinking.
The Sects created by this method had special names
related syntactically to the word UNIX.
Only XENIX
has survived as an example of this.

Then seeing all this activity, AT&T
decided to become a Vendor.

The Gurus were horrified when the Marketing Staff, regarded
today as the front line fighting force of all Corporations, were put
in charge of the development and promulgation of the cult.
The Creators were no longer allowed to directly influence the
development, instead they were to pass their ideas to the Marketeers
who would decide what was acceptable or not.
The Marketeers spent a lot of time producing a new religious tome
to help to guide the developers.
The book was known as the Svid and laid out the central
tenets of the practice.
The Marketeers were determined that the Svid should be elevated to
the level of all the other religious books.
To this end
they tried to make the Svid as cryptic as all the others, and succeeded.

After some initial shock, the Vendors accepted the central control
which AT&T
imposed because they saw that it
made their Products accessible to more Users.
`Consider it Standard' became the watchword in a Crusade designed
to eliminate the undesirable Sects which wished to differ from the Svid.

The Users of the Vendors Products
were certainly pleased with AT&T's
decision to become a Vendor.
It meant that they were no longer tied to the Products of one
particular Vendor but could pick and choose without altering their
worship.
The Gurus were less pleased until they realised that
AT&T were unable to change the cult to eliminate them.
Every Overlord where the UNIX
cult was practiced still required a Guru
to perform essential tasks.

The Sects

The UNIX
cult was always noted for its propensity to split into separate Sects.
In the early days, the Creators had a release policy which
made sure that all cult members possessed all the relevant facts
in order to fully comprehend the implications of the worship.
It was said that Users aspiring to be Gurus
needed all the facts because the religious books were written for Gurus,
and Users could make no sense of them.
Unfortunately, access to the information
was much abused because the Gurus immediately used
the knowledge to alter things and many
minor and major Sects of the Cult sprang into life.
The ability of each site to generate its own Sect was somewhat
curtailed by the cunning ploy of re-issuing the rituals and
practices in a slightly altered form from time to time.
The Gurus were soon tired of altering the same things
every time a new set of rituals came though the door and they began
to leave things alone.
Also, by this time, the Vendors had made an appearance and because they
believed in the Svid Crusade, they stuck with the orthodox mainstream
AT&T view of the cult.

Even so, at the end of the period being researched there seems to have been
two Sects with differing practices and rituals.
The main rival to the orthodox view
was a Guru lead Sect called the `Berkeley System Devotees'.
This had sprung into existence in the early days of the cult,
taking advantage of the knowledge imparted to them by the Creators.
However, the early Berkeley Gurus
cleverly distributed their rituals and practices
in the much the same way as the Creators and this ensured a wider
following.
The prominence was noted by a higher power and
they were chosen to master
the revisions of the Cult which were needed to permit worship on
some new Overlords.

These new Overlords had managed to conquer the restrictive memory sizes
which plagued many of their forbears.
It was said by many that the new Overlords had not got this right but at least
they did it.
As a result the new Overlords had the potential to allow bigger and
more expansive practices.
The Berkeley Gurus grasped this opportunity with the
objective of creating `The Perfect Ritual'.
The rival Sects referred to this as `creeping featurism';
the precise meaning of this obscure phrase is under investigation.

The Perfect Ritual was defined as one where all the letters of the
alphabet were used as a `parameter' specifying a distinct action
or phase in the worship.
The Berkeley Gurus never managed to create the Perfect Ritual,
but they came close.

The Berkeley Gurus distributed their rituals
and these became popular because the Marketeers inside AT&T
were so busy creating the Svid that they failed to notice that
Berkeley practices were slowly being adopted on all the new bigger Overlords.
The Berkeley rituals were also liked by Gurus because they were
nearer the `Old Religion' laid down by the Creators.
In a fit of pique, the AT&T
Marketeers decided that they would no longer
support the new Overlords and branded the Berkeley Gurus as heretics.

As heretics, the Berkeley Gurus decided to go one step further in
altering their Sect.
They proposed and executed a fundamental change of direction which
was to become a `De-facto Standard'.
The new Sect was revolutionary because it allowed Overlords to talk
to each other, but to do this a whole new litany had to be created.
New words entered the vocabulary and new concepts were introduced.
For many, worship in the new Sect was slow and unwieldy in comparison
to what had gone before.
But the new practices meant the easy
ability to interconnect Overlords and this was demanded by the Overlords
themselves.

Unfortunately for the
participants in the Svid Crusade, many Users actually insisted
on being able to use some of the Berkeley rituals.
The pressure from the Users was such that we begin to see Vendors
announcing their Products as being `with Berkeley enhancements'.
Finally, the AT&T
Marketeers were forced to incorporate some features of the
rituals which did not conflict with the teachings in the Svid.
At this time, we also begin to see specially created
Overlords which could be used to worship in the practices of either Sect simultaneously,
this was known as the `Universe Concept'.

The Berkeley Gurus were so broken by the gestation of the new Sect that many
left, some to worship the sun and
some to seek salvation in the noble task of Pixel Creation.
It was thought that the new Sect would be the last to
sally forth from hallowed halls of Berkeley because
the staff were demoralised and without joy.
The Svid Crusaders were pleased,
``All we have to do is wait and do nothing'', they said.
Since they weren't noted for doing much anyway, this wasn't difficult.

However, much to the dismay of the Crusaders,
many Novices amongst Berkeley group were promoted to Gurus,
and these new Gurus worked to consolidate and strengthen the new rituals.
The label of `slow and unwieldy' was not to be applied again.
Time and motion studies were performed on the rituals
for the first time in the recorded history of the
UNIX cult and a little more than lip service was paid to the notion
of efficiency of worship.

The Standardisers

One way of defeating the degeneration of the pure
UNIX cult into Sects was by the creation of a `Standard'.
As we have seen, the Svid is one example of this.
However, the Svid differed from other Standards because only the
AT&T marketeers were in a position to generate a Standard without reference
to anyone else.
They seemed especially keen that no taint of the Berkeley heresy should
appear in their work and so did not consult the Users.

In order that the Svid could qualify as a Standard, the
AT&T Marketeers had to promise that it would not alter.
They agreed to this because
they were determined to ensure that the Svid gained religious significance.
This was a good thing for other Vendors who were treating the
Svid as a Standard but it is possible that the slavish adherence
was detrimental to the fortunes of
AT&T in the long run because they were unable to update the rituals and
practices to keep up with the demand for change created by the Users.

All the other standards were either created by groups of Vendors
working together and ignoring the Users;
or by groups of Users working together and ignoring the Vendors.
Gurus were rarely involved; they were either too busy and
important to sit on committees or just plainly could not see the need for
conformity.

The Standards rarely reflected the
UNIX practices and rituals which were in use at the time of their creation.
All of them seemed to have a speculative element, as if the
Standardisers themselves tried to develop or perhaps rationalise the
rituals in some way.
As a result, the Standards were never standard.

However, the various Standards did give the Users an idea of what
was expected of them if they desired to move from one Sect to another.
Users who did this often, the so-called
Portables, learned to use
the minimum of ritual and to localise the Sect dependent areas of
their worship.

The Portables might have been helped by a Standard for the religious
language, C.
They were surprised to learn that Standard C was, in fact, a different
language from the original and almost no-one had an Overlord
which could understand it.
The changes were no doubt desirable but came from treating C as a
`high-level language' rather than using it in the way which the
Creators intended, a method of communicating intimately with
the Overlords.

Conclusion

The archives are still being searched for other interesting material
but enough has been found to demonstrate the fervent activity which
followed the creation of the
UNIX cult.
We have found no trace of the Creators and barely a hint of the
disciples who followed them inside AT&T .
We hope that
more research may provide some answers and respectfully
ask for more funding.




On to Sun Expert Articles








Peter Collinson Last change: 7th September 2008





, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Thursday, May 8, 2008

More Helpful Usage Commands For Solaris 10 Unix ZFS

Hey there,

Today we're here with part three of our four-part mini-series on ZFS and storage pool commands and tips. We started out with ZFS and storage pool creation commands, and followed that up with yesterday's post on maintenance and usage commands. Today's post is actually a continuation of yesterday's post, which was originally going to be part two of three. The inmates must be running the asylum ;)

For any notes pertaining to this entire four-post series, please refer to the first ZFS and storage pool post. I'm getting tired of hearing my own voice in my head ;)

Today, we'll keep on going with usage and maintenance tips. Enjoy and don't be afraid to come back for seconds :)

1. You can run statistics specifically against your ZFS storage pools (which can help in identifying a bottleneck on a system with a lot of virtual devices, zones and pools). These stats aren't too exciting since I just created the pool for this little cheat-sheet:

host # zpool iostat zvdevs 1 5
capacity operations bandwidth
pool used avail read write read write
---------- ----- ----- ----- ----- ----- -----
zvdevs 92K 1016M 0 0 83 633
<--- As with regular iostat, this is a summary line
zvdevs 92K 1016M 0 0 0 0
zvdevs 92K 1016M 0 0 0 0
zvdevs 92K 1016M 0 0 0 0
zvdevs 92K 1016M 0 0 0 0


2. If you ever need to list out your ZFS file systems, just use the "list" option, like this:

host # zfs list |grep zvdevs
zvdevs 120K 984M 26.5K /zvdevs
zvdevs/vusers 24.5K 984M 24.5K /zvdevs/vusers


3. One problem you might note with new ZFS files systems and listing them out, is that they all have access to the entire storage pool. So, theoretically, any of them could fill up the entire storage pool space and leave the other two out of disk space. This can be fixed by setting "reservations" to ensure that each VFS file system gets to keep a certain amount of space for itself no matter what (here, vusers gets at least 100 Megabytes, and vusers2 and 3 will get at least 50 Megabytes each):

host # zfs set reservation=100m zvdevs/vusers
host # zfs set reservation=50m zvdevs/vusers2
host # zfs set reservation=50m zvdevs/vusers3
host # zfs list -o name,reservation|grep zvdevs
zvdevs none
zvdevs/vusers 100M
zvdevs/vusers2 50M
zvdevs/vusers3 50M


4. On the other hand, you may want to restrict space-hogs from taking up too much space. You can do this with quota's, almost just like on older versions of Solaris Unix:

host # zfs set quota=200m zvdevs/vusers
host # zfs set quota=100m zvdevs/vusers2
host # zfs set quota=100m zvdevs/vusers3
host # zfs list -o name,quota|grep zvdevs
zvdevs none
zvdevs/vusers 200M
zvdevs/vusers2 100M
zvdevs/vusers3 100M


5. You can also set up file systems to use compression, selectively, like so (Whether or not this is worthwhile is debatable):

host # zfs set compression=on zvdevs/vusers2
host # zfs list -o name,compression|grep zvdevs
zvdevs off
zvdevs/vusers off
zvdevs/vusers2 on
zvdevs/vusers3 off


6. ZFS also has built-in snapshot capabilities, so you can prep yourself (like, before you're about to try something crazy ;) and be able to rollback to an earlier point in time (We'll call our snapshot "backup"), like this:

host # zfs snapshot zvdevs/vusers@backup
host # zfs list|grep zvdevs
zvdevs 200M 784M 28.5K /zvdevs
zvdevs/vusers 24.5K 200M 24.5K /zvdevs/vusers
zvdevs/vusers@backup 0 - 24.5K -
zvdevs/vusers2 24.5K 100M 24.5K /zvdevs/vusers2
zvdevs/vusers3 24.5K 100M 24.5K /zvdevs/vusers3


7. Now, assuming that you screwed everything up so badly that you need to back out your changes (but not so bad that you can't access Solaris ;), you can use ZFS's built-in rollback capability. Here we'll do a rollback, based on our snapshot, and clone that to a separate ZFS file system, so we can move back whatever files we need to their original locations (Unfortunately, the extra cloning steps are necessary, as snapshots are not directly accessible and you also can't clone to an existing vdev! ):

host # zfs rollback zvdevs/vusers@backup
host # zfs clone zvdevs/vusers@backup zvdevs/vusers
cannot create 'zvdevs/vusers': dataset already exists
<--- Didn't I just remind myself that I can't do this ;)
host # zfs clone zvdevs/vusers@backup zvdevs/vusers4
host # zfs list|grep zvdevs
zvdevs 200M 784M 29.5K /zvdevs
zvdevs/vusers 24.5K 200M 24.5K /zvdevs/vusers
zvdevs/vusers@backup 0 - 24.5K -
zvdevs/vusers2 24.5K 100M 24.5K /zvdevs/vusers2
zvdevs/vusers3 24.5K 100M 24.5K /zvdevs/vusers3
zvdevs/vusers4 0 784M 24.5K /zvdevs/vusers4


8. Another cool thing you can do is to rename your file systems, like this (note that any attached snapshots will be renamed as well):

host # zfs rename zvdevs/vusers4 zvdevs/garbage
host # zfs list|grep zvdevs
zvdevs 100M 884M 28.5K /zvdevs
zvdevs/garbage 24.5K 884M 24.5K /zvdevs/garbage
zvdevs/garbage@backup 0 - 24.5K -
zvdevs/vusers2 24.5K 100M 24.5K /zvdevs/vusers2
zvdevs/vusers3 24.5K 100M 24.5K /zvdevs/vusers3


9. You can also rename snapshots, if you don't want anybody (including yourself in a few months ;) to be able to easily tell what file system is a backup of what other file system, like this:

host # zfs rename zvdevs/garbage@backup zvdevs/garbage@rollbacksys
host # zfs list|grep zvdevs
zvdevs 100M 884M 28.5K /zvdevs
zvdevs/garbage 24.5K 884M 24.5K /zvdevs/garbage
zvdevs/garbage@rollbacksys 0 - 24.5K -
zvdevs/vusers2 24.5K 100M 24.5K /zvdevs/vusers2
zvdevs/vusers3 24.5K 100M 24.5K /zvdevs/vusers3


10. And, just in case you need to have all the information you could ever want about your ZFS file systems, you can "get" it "all" :

host # zfs get all zvdevs
NAME PROPERTY VALUE SOURCE
zvdevs type filesystem -
zvdevs creation Mon May 5 10:00 2008 -
zvdevs used 100M -
zvdevs available 884M -
zvdevs referenced 28.5K -
zvdevs compressratio 1.00x -
zvdevs mounted yes -
zvdevs quota none default
zvdevs reservation none default
zvdevs recordsize 128K default
zvdevs mountpoint /zvdevs default
zvdevs sharenfs off default
zvdevs checksum on default
zvdevs compression off default
zvdevs atime on default
zvdevs devices on default
zvdevs exec on default
zvdevs setuid on default
zvdevs readonly off default
zvdevs zoned off default
zvdevs snapdir hidden default
zvdevs aclmode groupmask default
zvdevs aclinherit secure default
zvdevs canmount on default
zvdevs shareiscsi off default
zvdevs xattr on default


11. It's better to send than to receive :) Maybe not... In any event, you can use the ZFS send and receive commands to send snapshots to other filesystems locally or remotely (much in the same way you can use the system subshell to move content using tar), like so:

host # zfs send zvdevs/vusers@garbage | ssh localhost zfs receive zvdev/newbackup
host # zfs list
host # zfs list|grep zvdevs
zvdevs 100M 884M 28.5K /zvdevs
zvdevs/garbage 24.5K 884M 24.5K /zvdevs/garbage
zvdevs/vusers@backup 0 - 24.5K -
zvdevs/vusers2 24.5K 100M 24.5K /zvdevs/vusers2
zvdevs/vusers3 24.5K 100M 24.5K /zvdevs/vusers3
zvdevs/newbackups 24.5K 100M 24.5K /zvdevs/newbackup


12. If you get nostalgic from time to time, or you need to figure out why something bad happened, you can use the ZFS filesystem history command, like so (Note that no history will be available after you destroy a pool -- I did that already, to zvdevs, so this output is for another storage pool on the same system):

# zpool history
History for 'datadg':
2008-02-27.20:25:32 zpool create -f -m legacy datadg c0t1d0
2008-02-27.20:27:57 zfs set mountpoint=legacy datadg
2008-02-27.20:27:58 zfs create -o mountpoint -o quota=5m datadg/data
2008-02-27.20:27:59 zfs create -V 20gb datadg/swap
2008-02-27.20:31:47 zfs create -o mountpoint -o quota=200m datadg/oracleclient


Cheers,

, Mike

Monday, December 3, 2007

Finding User Accounts To Disable - Follow-Up Post

Hey there,

This is a follow up to a previous post where we're walking through the process of developing a decent script, from scratch; examining how to figure out what user accounts you could disable or delete (times vary, depending on your company's policy) on any particular box you have to administrate.

Today I've taken it a step farther. At this point the below script will check the modification time of all users listed in /etc/passwd and compare it against the current Unix time. If the mtime on the home directory is over 45 days and less than 90, we warn to disable (actual code not included) and if it's over 90 days, we warn to delete (same "no code" policy for that ;). If the time on the home directory is under 45 days, the account is consider active and OK.

Note that I found a few incorrect things and changed them, as I augmented this script; so this isn't just a bigger version of the original. First of all, it seems that the user's home directory is just as reliable as the existence of a history file. Also, it's much more likely to exist ;) Also, by verifying, I noticed that the mtime (Last Modification Time) was more reliable than the atime (Last Access Time) that we used before. That seems a bit counter-intuitive, but so far, it's proving out.

We've also added some extra functionality to remove users that we don't want to check. The first group are in a file we've called locked.txt, the second group are users without a valid shell and the final group are users who are in a particular group (like administrators who need accounts on all boxes, but might not login to any particular one for a long time).

Hope this script helps, at least, to point a few people in the right direction. This is a work in progress since it still contains several flaws, including no cross-referencing of possibly-available data that the system can provide with "last" and other such commands, as well as still having to deal with "touch."

Note that, on the lines ---> @locked_text = ###LOCKEDTEXT###;
and
open(LOCKEDTEXT, "###locked.txt");

the ### space-holders should be greater than and equal signs <>, in the first instance, and a < sign in the second, but the blog software interprets them as a unclosed and unallowed tags!

Have fun :)


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/perl

#
# 2007 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
# switching to mtime 9 - from atime - 8
#
# We don't need to check for errors on the command line since
# we won't be accepting any arguments
#

if ( -f "moreinfo") {
print "Removing old temporary file \"moreinfo\"\n";
unlink("moreinfo");
}

$today=time();

@no_shells=`egrep -i 'nologin|false' /etc/passwd|awk -F":" '{print \$1}`;
@in_unixtech=`grep 1621 /etc/passwd|awk -F":" '{print \$1}`;
@user_names=`awk -F":" '{print \$1}' /etc/passwd`;
open(LOCKEDTEXT, "###locked.txt");

@locked_text = ###LOCKEDTEXT###;
close(LOCKEDTEST);
print "\nRemoving LOCKED ACCOUNTS ";
foreach $locked_user (@user_names) {
print ".";
chomp($locked_user);
if ( grep /$locked_user/,@locked_text ) {
push(@tmp_user_names,$locked_user);
print "$locked_user ";
}
}
print "\n";
print "\nRemoving NO_SHELL ACCOUNTS ";
foreach $locked_user (@user_names) {
print ".";
chomp($locked_user);
if ( grep /$locked_user/,@no_shells ) {
push(@tmp_user_names,$locked_user);
print "$locked_user ";
}
}
print "\n";
print "\nRemoving UNIXTECH ACCOUNTS ";
foreach $locked_user (@user_names) {
print ".";
chomp($locked_user);
if ( grep /$locked_user/,@in_unixtech ) {
push(@tmp_user_names,$locked_user);
print "$locked_user ";
}
}
print "\n\n";
@total = @both = @unique_names = ();
%count = ();
foreach $possible_dupe (@user_names, @tmp_user_names) {
$count{$possible_dupe}++
}
foreach $possible_dupe (keys %count) {
push @total, $possible_dupe;
push @{ $count{$possible_dupe} > 1 ? \@both : \@unique_names }, $possible_dupe;
}
@user_names = @unique_names;
foreach $diruser (@user_names) {
$filer = `grep $diruser /etc/passwd|awk -F":" '{print \$6}'`;
push(@user_dirs,$filer);
}
print "FIRST RUN - Checking Home Directories...\n\n";
foreach $history_file (@user_dirs) {
chomp($history_file);
chomp($user_name = $user_names[$counter]);
print "${user_name}: ";
$counter++;
if ( -d "$history_file" ) {
@file_dates=stat("$history_file");
$file_date=$file_dates[9];
} else {
$file_date=0;
}
$difference=$today - $file_date;
if ( $difference >= 3888000 && $difference < 7776000 ) {
print "User needs to be disabled - ";
print "45 days old or older, but under 90\n";
# YOUR CUSTOM CODE HERE - "passwd -l" would probably suffice"
} elsif ( $difference >= 7776000 ) {
print "User needs to be deleted - ";
print "90 days old or older\n";
# YOUR CUSTOM CODE HERE -
} else {
print "OK\n";
}
}


, Mike





Saturday, December 1, 2007

Disabling and Removing Old User Accounts

Depending upon the size of your network, keeping track of user accounts on all of your Unix and Linux boxes can range from fairly simple to unbelievably hard to keep track of. I've noticed on the internet boards that lots of people are looking for the "one" way to semi-automate determining if user accounts need to be disabled, or if they're so old they need to be deleted.

I'm not a believer in the "one" solution theory. One of the things I love about Linux and Unix is, depending on how creative you're willing to be, there are probably 50 or more ways to do anything.

In any event I started looking at this problem, and noticed that almost everyone was using the "finger," "logins" or "last" commands to figure out how long it had been since someone logged in. In some extreme cases, I've even found examples where folks "pack" and "unpack" /var/adm/wtmpx with Perl to get the information. So, I naturally thought: Is it possible to do this without using the system's login-logging facilities?

The more I thought about this, the more I came to understand that both methods had their virtues and their faults. For instance, if /var/adm/wtmpx is missing, a lot of regular system accounting programs won't work. If you corrupt that file and remove /var/adm/lastlog (I'm using Solaris as my model - substitute file locations and names as necessary), "logins," "finger" and "last" don't report correctly unless they're reporting an error. The alternative approach also has its flaws. If, for instance, a user utilizes SSH to run a single command from a remote host using key-based passworless login, that shell won't be interactive and it won't update any shell history files!

Tackling this problem will take more than one post, so we'll start out with the core; treat this as a sort of tutorial on the phases that you go through from scratch to a decent script. Our assumption here will be that we have no access to any of the regular utilities like "last," "finger" and "logins." We'll use Perl to check out all users' .sh_history or .bash_history files (just covering sh, ksh and bash for now) and use the "stat" and "time" functions to determine how long it's been since the user last logged in.

Here's the core of the code, to get started. I've written it so it can be run on its own for now:


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/perl

#
# 2007 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
#
# We don't need to check for errors on the command line since
# we won't be accepting any arguments
#

$today=time();

@user_dirs=`awk -F":" '{print \$6}' /etc/passwd`;
foreach $history_file (@user_dirs) {
chomp($history_file);
print "${history_file}: ";
if ( -f "$history_file/.sh_history" ) {
@file_dates=stat("$history_file/.sh_history");
$sh_file_date=$file_dates[8];
} else {
$sh_file_date=0;
}
if ( -f "$history_file/.bash_history" ) {
@file_dates=stat("$history_file/.sh_history");
$bash_file_date=$file_dates[8];
} else {
$bash_file_date=0;
}
if ( $sh_file_date > $bash_file_date ) {
$file_date = $sh_file_date;
} elsif ( $bash_file_date > $sh_file_date ) {
$file_date = $bash_file_date;
} else {
print "Cannot find .bash_history or .sh_history for $history_file. Moving on!\n";
next;
}
$difference=$today - $file_date;
if ( $difference >= 3888000 && $difference < 7776000 ) {
print "User needs to be disabled- ";
print "45 days old or older, but under 90\n";
# YOUR CUSTOM CODE HERE - "passwd -l" would probably suffice"
} elsif ( $difference >= 7776000 ) {
print "User needs to be deleted - ";
print "90 days old or older\n";
# YOUR CUSTOM CODE HERE -
} else {
print "OK\n";
}
}


Within the next post or two, we'll follow up on this, so that we can add functionality to check system files, using system utilities, to verify that our initial information is correct. It's possible that someone just "touch"ed a user's history file and that access time information isn't necessarily correct. Also, as we noted above, folks can be using the host remotely, in a non-interactive shell, and never alter the access time of their history file.

For the time being, this code should be a good start, and something to think about and improve upon. Until next time :)

, Mike