Showing posts with label number. Show all posts
Showing posts with label number. Show all posts

Tuesday, December 2, 2008

Predicting Solaris 10 TCP Sequence Numbers Part 1: Initial Discovery

Hey there,

Note: All examples today were produced using Solaris 10. As such, commands like snoop may need to be replaced with ethereal or tcpdump, etc, depending on your OS and/or preference.

Today, we're going to look at a subject that gets plenty of attention (but, maybe not as much publicity) in the world of Unix and Linux network computing: TCP packet sequence number prediction, how it's used to protect network transmissions and whether or not, with the advent of packet checksumming, etc, it's even a factor in basic network security anymore. See here, or at the very bottom of the post, for something potentially interesting about the latest Solaris' network protocol enhancements including SDP.

Not all that long ago (which might mean a long time ago to you, since I'm growin' old ;) TCP number sequencing was pretty simple to predict. In essence, the sequence numbering of TCP packets is required since TCP does its own error-correction and will attempt to resend packets that it doesn't "know" reached their specified endpoint. Numbering, at the most basic level, was/is a great way to ensure that all packets sent actually get received. And, believe it or not, at one point in time that numbering was simply incremental. That means that, given a point-to-point TCP transaction, a simple transmission (from the sender's point-of-view, for instance) might consist of 15 packets. If the first packet was numbered 1 (a gross oversimplification, to be sure, but not too far from being exact ;), the next would be numbered 2, then 3 and so forth until the 15th packet (numbered 15) was successfully sent.

Given the above scenario, and with the aid of hindsight, TCP connection hijacking and other spoofing-based attacks were made quite a bit simpler than they might have been. Of course, once all this "bad" activity began happening, the specifications for TCP packet sequence numbering began to change and become more complicated. Many proprietary OS manufacturer's also began to implement their own proprietary sequence numbering algorithms (in accordance with standards so they could communicate with "unlike" systems). Unfortunately, most of these next-gen specifications were based on simplistic rules like (another gross oversimplification) simple mathematic derivations which didn't use any sort of "outside influence" to affect (or make more unpredictable) the sequence numbering. If you could figure out the equation, you could easily guess what packet number would be next, given one or two sequential packets, and then the whole mess would start all over again. Wikipedia actually has a very nice page on TCP with links to all the relevant RFC's.

This is actually still true today, although OS manufacturers, the Open Source community and white/black-hat hackers are stepping up their games to greater degrees and at a much more condensed pace (by which I mean things now, generally, change more often within a shorter period of time - another gross oversimplification ;)

So, now that it's the present, let's take a look at Solaris 10 and check out what they're doing with regard to this matter (From release 10/08 - the most recent to this date, as far as I know). Assuming we have a connection established with a Solaris 10 server (simple SSH), we can use snoop to gather the information we need to even begin attempting to figure out the TCP packet sequence numbering algorithm they're using (assuming - thinking positive thoughts :) that we can. A simple chunk of output is shown below:

host # snoop -r -d eri0 from 10.99.99.99 to 10.98.98.98
Using device /dev/eri (promiscuous mode)
10.99.99.99 -> 10.98.98.98 TCP D=2782 S=22 Push Ack=2865625835 Seq=673983919 Len=52 Win=50400
10.99.99.99 -> 10.98.98.98 TCP D=2782 S=22 Push Ack=2865625887 Seq=673983971 Len=52 Win=50400
10.99.99.99 -> 10.98.98.98 TCP D=2782 S=22 Push Ack=2865625939 Seq=673984023 Len=52 Win=50400
10.99.99.99 -> 10.98.98.98 TCP D=2782 S=22 Push Ack=2865625939 Seq=673984075 Len=340 Win=50400
10.99.99.99 -> 10.98.98.98 TCP D=2782 S=22 Push Ack=2865625939 Seq=673984415 Len=148 Win=50400
10.99.99.99 -> 10.98.98.98 TCP D=2782 S=22 Push Ack=2865625939 Seq=673984563 Len=148 Win=50400
10.99.99.99 -> 10.98.98.98 TCP D=2782 S=22 Push Ack=2865625939 Seq=673984711 Len=148 Win=50400
10.99.99.99 -> 10.98.98.98 TCP D=2782 S=22 Push Ack=2865625939 Seq=673984859 Len=148 Win=50400
10.99.99.99 -> 10.98.98.98 TCP D=2782 S=22 Push Ack=2865625939 Seq=673985007 Len=148 Win=50400
10.99.99.99 -> 10.98.98.98 TCP D=2782 S=22 Push Ack=2865625939 Seq=673985155 Len=148 Win=50400


Of course, the column we're most interested in (at first, anyway) is the 9th column from the left (assuming spaces and tabs as the default field separator). The column beginning with "Seq=" is what we'll look at first. For the sake of this exercise, we've saved all the packets in a file called "packets." We didn't use -o on the command line, but this is a pretty simple cut-and-paste ;). To isolate those sequence numbers, we could just do the following:

host # awk '{print $9}' packets|sed 's/Seq=//'
673983919
673983971
673984023
673984075
673984415
673984563
673984711
673984859
673985007
673985155


And, here's where it becomes difficult. The first test of a reasonably stable (by which I mean, unpredictable) sequence numbering algorithm is to subject it two basic tests. That is to say, is it really secure or is it just a simple numeric or mathematic progression in sheep's clothing?

As we bring part 1 of this endeavour to an end, we'll do the quick and dirty testing (If Solaris fails these, then I may have made a big mistake when I named this post before I finished writing it ;)

1. Are the numbers in simple sequence (e.g. +1 or +2) over and over again?
Answer: No. The numeric difference between the packet pairs (1-2, 2-3, etc) are not infinitely exact.
Proof: The 9 differences in order:
673983971-673983919 = 52
673984023-673983971 = 52
673984075-673984023 = 52
673984415-673984075 = 340
673984563-673984415 = 148
673984711-673984563 = 148
673984859-673984711 = 148
673985007-673984859 = 148
673985155-673985007 = 148


2. Is there any direct correlation between the differences between packet sequence numbers and any other property of the packets?
Answer: Yes. If we scroll up the page and look at the basic snoop output, we can see that the difference in sequence numbers correlates exactly with the length of the packet.
Proof: The packet lengths in order:
host # awk '{print $10}' packets|sed 's/Len=//'
52
52
52
340
148
148
148
148
148
148


It seems as though we have something going here, but it must be harder than this to guess the sequence numbers. It's possible, also, that additional measures are being used that obviate the somewhat-flimsy dependence on convoluted number sequences to safeguard a transmission from "packet injection" or hijacking. For instance, Solaris 10 now makes use of SDP as part of its data transmission security (see, also, the Sockets Direct Protocol Manpage). It may also be that Devil is in the details and we only looked at a surface scan of our packets. We'll look at all of these possibilities in a near-future post. This isn't over yet :)

Cheers,

, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Thursday, September 18, 2008

Streamlined Perl Number Matching Script For Unix Or Linux

Hey There,

We'll, this week's been completely out-of-sequence. Today's post was slated for the day before yesterday and Tuesday's number pool script that was supposed to come out yesterday ended up getting published on Tuesday. If there's one thing we've learned this week, it's that my day job is starting to interfere with my hobbies ;)

On the bright side, we're finally getting around to wrapping up the two-parter we started on Monday with our uncondensed Number Matching Perl Script. The intent, with that post, was to show a working example of a simple Perl script (basically, matching numbers out of another file with a few option-based limitations imposed by the user) that was written off-the-cuff. Although it functioned, and worked pretty efficiently, it was a confusing mess to look at, and didn't include some options that it probably should have. To summarize, the aim of the follow up post (today's, at last) was going to be to reproduce that script in a more condensed format. This condensation (that just doesn't sound right, does it? But, I looked it up in the dictionary and the way in which I meant it was actually the first entry ;) would include addition of extra code to handle some needed functionality, as well as a few helpful comments and more logical ordering of all the parts. The end result, hopefully, is more scalable, so you won't have to dread all the extra typing and logical convolution involved in adding a seventh variable ;)

Again, we have the same fairly simple Perl script that will parse any file consisting of rows of numbers and print out matches meeting certain criteria. As noted before, you can visit the lamer version of this script from Monday and compare it with today's "better" version (or "less lame," depending on what kind of a person you are. Glass half full? Glass half empty? Glass too big? ;)

Again, the basic file format (for the input) should be a text file with lines like the following (note that this updated script will also take regular single digits and has an extra "-p" flag if you want to pad those with 0's):

host # cat file
01 02 03 04 05 09
5 18 19 45 33
33 55 666 88 23 12
...



Again, the script has 4 non-optional arguments that you'll be prompted for if you forget to enter any of them (with the additional, and optional, "-p" flag), like so:

host # ./match.pl
Error Encountered! Invalid or incomplete options!
Usage: ./better.pl -h highNumber -f statFile -n numberOfCombos -m mimimumCombos
[-p padSingleDigitsWithZeros]


A regular execution would look like this (we're removing duplicate match elements in a simple hash now, instead of building an increasingly lengthening logic knot - depending upon how you want to implement the match, it should be trivial, like adding \b or whatnot, to make the matches more exact so that 13 won't match 130):

host # ./match.pl -h 15 -f NUMFILE -n 3 -m 10 -p
12 matches 10 times:
7 8 9 10 11 12
109 110 111 112 113 114
115 116 117 118 119 120
121 122 123 124 125 126
127 128 129 130 131 132
211 212 213 214 215 216
307 308 309 310 311 312
409 410 411 412 413 414
511 512 513 514 515 516
607 608 609 610 611 612

13 matches 10 times:
13 14 15 16 17 18
109 110 111 112 113 114
127 128 129 130 131 132
133 134 135 136 137 138
139 140 141 142 143 144
211 212 213 214 215 216
313 314 315 316 317 318
409 410 411 412 413 414
511 512 513 514 515 516
613 614 615 616 617 618

15 matches 10 times:
13 14 15 16 17 18
115 116 117 118 119 120
145 146 147 148 149 150
151 152 153 154 155 156
157 158 159 160 161 162
211 212 213 214 215 216
313 314 315 316 317 318
415 416 417 418 419 420
511 512 513 514 515 516
613 614 615 616 617 618


And, when everything isn't equal, the output basically looks the same :)

host # ./match.pl -h 15 -f NUMFILE -n 3 -m 5 -p

01 02 matches 6 times:
97 98 99 100 101 102
199 200 201 202 203 204
301 302 303 304 305 306
397 398 399 400 401 402
499 500 501 502 503 504
601 602 603 604 605 606...


All in all, we only save about 50 lines of code (some of which is blank lines and comments), but the result should be a lot easier to manage and build upon.

Hope you enjoy the script and can find some use for it or its parts (like dealing with duplicates in multiple arrays, etc). You can never do enough matching ;)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/usr/bin/perl

#
# match.pl - Updated to last longer and taste better :)
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

use Getopt::Std;

%options = ();

# DIFF - Add -p option to pad 1-9
getopts("h:f:m:n:p", \%options);

if ( defined $options{m} && defined $options{h} && defined $options{f} && defined $options{n} ) {
$highnum = $options{h};
$combos = $options{n};
$statfile = $options{f};
$minmatch = $options{m};
if ( ! -f $statfile ) {
usage("File $options{f} Does Not Exist!");
} elsif ( $combos > 6 ) {
usage("Only Combos Up To 6 Please!");
}
} else {
usage("Invalid or incomplete options!");
}


open(FILE, "<$statfile");
@file = <FILE>;
close(FILE);

# DIFF - with -p, now have to differentiate
# DIFF - between straight and padded 1-9
for ( $lownum = 1; $lownum <= $highnum; $lownum++ ) {
if ( defined $options{p} && $lownum < 10 ) {
$padded_num = "0$lownum";
} else {
$padded_num = $lownum;
}
push(@numbers, "$padded_num");
}

# DIFF - Still not using a subroutine for this.
# Trying to stay away from arrays of array references, also
# for ease of understaning

@numbers1 = @numbers2 = @numbers3 = @numbers4 = @numbers5 = @numbers6 = "";
$one = $two = $three = $four = $five = $six = 1;
$start = 1;
while ( $start <= $combos ) {
if ( @numbers1 == 1 && $start == 1 ) {
@numbers1 = @numbers;
$one = 0;
} elsif ( @numbers2 == 1 && $start == 2 ) {
@numbers2 = @numbers;
$two = 0;
} elsif ( @numbers3 == 1 && $start == 3 ) {
@numbers3 = @numbers;
$three = 0;
} elsif ( @numbers4 == 1 && $start == 4 ) {
@numbers4 = @numbers;
$four = 0;
} elsif ( @numbers5 == 1 && $start == 5 ) {
@numbers5 = @numbers;
$five = 0;
} elsif ( @numbers6 == 1 && $start == 6 ) {
@numbers6 = @numbers;
$six = 0;
}
$start++;
}

foreach $cnum1 (@numbers1) {
foreach $cnum2 (@numbers2) {
foreach $cnum3 (@numbers3) {
foreach $cnum4 (@numbers4) {
foreach $cnum5 (@numbers5) {
foreach $cnum6 (@numbers6) {
@match = grep(/$cnum1/ && /$cnum2/ && /$cnum3/ && /$cnum4/ && /$cnum5/ && /$cnum6/, @file);
$match = @match;
if ( $match >= $minmatch ) {
undef %seen;
@success = grep(!$seen{$_}++, ($cnum1, $cnum2, $cnum3, $cnum4, $cnum5, $cnum6));
print "@success matches $match times:\n @match\n";
}
}
}
}
}
}
}

sub usage {
$message = shift;
print "Error Encountered! $message\n";
print "Usage: $0 -h highNumber -f statFile -n numberOfCombos -m mimimumCombos\n";
print "[-p padSingleDigitsWithZeros]\n";
exit(1);
}


, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Wednesday, September 17, 2008

Revamped Perl Script To More Evenly Distribute Number Pool Match Odds

Hey There,

It's been noted (and I know that it's true) that the spread-system used in the last portion of our Perl script (posted yesterday in our entry regarding maximizing guaranteed matches) was somewhat uneven. The end result, of course, being that your given set-numbers from pool-numbers (3 from 5, 4 from 6, etc) guaranteed that the sets would be included in the generated pools, but (as was noticed by a few people, including myself) the downside (?) ends up being that the front-numbers are more heavily weighted. That is to say that, given a situation where you've got 6 balls drawn, you want to guarantee matches of 3 and you're using a number pool of 8 different numbers (1, 2, 3, 4, 5, 6, 7 and 8 - for example), the first 3 return a higher percentage of the time than the numbers at the other end of the spectrum.

I actually chose to go with this sort of distribution for a particular reason. Well, reasoning, anyway. My reasoning was this: If I rewrite the distribution so that it ends up being more "fair" (The best I could get was a somewhat even distribution that would guarantee any given match set would come up only once or twice), this would probably seem like a better deal, since the odds are "spread out." This is where I, possibly, view things a little bit differently. In our original setup, even though the front numbers are more heavily weighted, the end numbers are guaranteed to contain match sets that all come up at least once or twice. Really, you've got the same thing as the more "fair" method, except for two strong distinctions (aside from the fact that the distributions are unequal in comparative outcomes):

1. In the "fair" distribution, your end number matched-sets have approximately the same odds of returning as they do in the "unfair" distribution. In the "fair" distribution, your end numbers tend to go back and forth between 1 and 2 match-sets guaranteed. In the "unfair" distribution, your end numbers are almost always going to be only 1 match set guaranteed. In neither case should you ever be getting back 0 match sets.

2. In the "fair" distribution, your front number matched-sets have the same odds of returning as your end number matched sets (1, and no more than 2 times guaranteed). In the "unfair" distribution, you'll find several match sets that will return matched sets over 10 times, and then a few more at 9 and some more at 8, a good amount of 4's and 3's and then down to 1's. Basically, your front numbers will be weighted to match not "more often," but "more heavily" if they do hit.

The heavy distribution differential can be easily drawn out, like this (using currency as the currency ;):

a. Front numbers.

1. Fair distribution: Matched set 123 is drawn. We match it twice. Each payout on a matched set it 5 dollars. We've made 10 dollars.
2. Unfair distribution: Matched set 123 is drawn. We match it 11 times. Each payout on a matched set is 5 dollars. We've made 55 dollars.

b. End numbers

1. Fair distribution: Matched set 789 is drawn. We may match it twice (rather than just once). Each payout on a matched set it 5 dollars. We've made 10 dollars.
2. Unfair distribution: Matched set 789 is drawn. We match it once. Each payout on a matched set it 5 dollars. We've made 5 dollars.

So, yes, I have to agree that a difference does exist (and would actually love to find the most economic way to make both ends of the distribution weigh in at 8 to 11 returns without having to run up too many combinations (At a certain point your expenses outweigh your gains... not to mention that those gains are only "possible" and not real until you, maybe, get lucky ;)

Although we went over this previously, I'll repost some data, with slight variations, so that the full code for both the "fair" and "unfair" distributions are covered.

Enjoy :)



FAIR:


@ball_draw_pool = qw(12345 12346 12347 12348 12356 12357 12358 12367 12368 12378 12456 12457 12458 12467 12468 12478 12567 12568 12578 12678 13456 13457 13458 13467 13468 13478 13567 13568 13578 13678 14567 14568 14578 14678 15678 23456 23457 23458 23467 23468 23478 23567 23568 23578 23678 24567 24568 24578 24678 25678 34567 34568 34578 34678 35678 45678);
@rev_ball_draw_pool = qw(45678 35678 34678 34578 34568 34567 25678 24678 24578 24568 24567 23678 23578 23568 23567 23478 23468 23467 23458 23457 23456 15678 14678 14578 14568 14567 13678 13578 13568 13567 13478 13468 13467 13458 13457 13456 12678 12578 12568 12567 12478 12468 12467 12458 12457 12456 12378 12368 12367 12358 12357 12356 12348 12347 12346 12345);

@ball_trap_pool = qw(123 124 125 126 127 128 134 135 136 137 138 145 146 147 148 156 157 158 167 168 178 234 235 236 237 238 245 246 247 248 256 257 258 267 268 278 345 346 347 348 356 357 358 367 368 378 456 457 458 467 468 478 567 568 578
678);

foreach $smaller (@ball_trap_pool) {
@smaller = split("",$smaller);
foreach $larger (@ball_draw_pool) {
print "$larger =~ $smaller[0] && $larger =~ $smaller[1] && $larger =~ $smaller[2]\n";
if ( $larger =~ $smaller[0] && $larger =~ $smaller[1] && $larger =~ $smaller[2] ) {
$total_pool{$larger} .= $smaller;
last;
$|=1;
}
}
}
foreach $smaller (@ball_trap_pool) {
@smaller = split("",$smaller);
foreach $larger (@rev_ball_draw_pool) {
print "$larger =~ $smaller[0] && $larger =~ $smaller[1] && $larger =~ $smaller[2]\n";
if ( $larger =~ $smaller[0] && $larger =~ $smaller[1] && $larger =~ $smaller[2] ) {
$total_pool{$larger} .= $smaller;
last;
$|=1;
}
}
}
$counter = 1;
foreach $key (sort keys %total_pool ) {
$total_poolkey = length $total_pool{$key};
if ( $total_poolkey <= 6 ) {
next;
}
print "Game $counter: $key -- $total_pool{$key}\n";
$counter++;
}



UNFAIR


@ball_draw_pool = qw(12345 12346 12347 12348 12356 12357 12358 12367 12368 12378 12456 12457 12458 12467 12468 12478 12567 12568 12578 12678 13456 13457 13458 13467 13468 13478 13567 13568 13578 13678 14567 14568 14578 14678 15678 23456 23457 23458 23467 23468 23478 23567 23568 23578 23678 24567 24568 24578 24678 25678 34567 34568 34578 34678 35678 45678);

@ball_trap_pool = qw(123 124 125 126 127 128 134 135 136 137 138 145 146 147 148 156 157 158 167 168 178 234 235 236 237 238 245 246 247 248 256 257 258 267 268 278 345 346 347 348 356 357 358 367 368 378 456 457 458 467 468 478 567 568 578 678);

foreach $smaller (@ball_trap_pool) {
@smaller = split("",$smaller);
foreach $larger (@ball_draw_pool) {
print "$larger =~ $smaller[0] && $larger =~ $smaller[1] && $larger =~ $smaller[2]\n";
if ( $larger =~ $smaller[0] && $larger =~ $smaller[1] && $larger =~ $smaller[2] ) {
$total_pool{$larger} .= $smaller;
last;
$|=1;
}
}
}
$counter = 1;
foreach $key (sort keys %total_pool ) {
$total_poolkey = length $total_pool{$key};
if ( $total_poolkey <= 6 ) {
next;
}
print "Game $counter: $key -- $total_pool{$key}\n";
$counter++;
}


, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Tuesday, September 16, 2008

Perl Script To Maximize Guaranteed Combinations Within Fixed Lists

Hey there,

Breaking tradition (or continuing it, depending on how often you read this blog ;) we're going to have part 2 of yesterday's two-parter tomorrow and put up our final Perl script to find the maximum guaranteed number sets, or lists, within larger number pools. There's not too much else to explain about it, but we will lay out this road map, so that you can see (or possibly revisit or avoid ;) all of the different topics we covered that are included in this script and/or are directly relevant (kind of a list of directions to this post and its attached script).

In order to get to this script, we've tackled:

1. Number Pools And Guaranteed Matches
2. Permuting Lists and Strings With Perl
3. Sorting Lists And Removing Duplicates With Perl
4. Unique Sorting Of Lists Within Lists Within Lists
5. Determining The Maximum Amount Of Pool Sets In Any Number Pool
6. Finding Overlapping Matches Using Lookahead Assertions
7. Evaluating Number Grids
8. Maximizing Number Set Probability Within Number Pools


And, so, without further ado, the culmination of all that effort (okay, that's a bit of a stretch... how about, the amalgamation of all those different concepts ;) - A simple Perl script that gets all its input interactively, so it can be run very simply, like so (sample run):

host # ./wheel.pl
Highest Number: 39
Number Of Balls Drawn: 5
How Many Numbers To Trap: 3
Enter your numbers
separated by space character : 1 2 3 4 5 6 7 19


Highest Number 39
Balls Drawn 5
How Many Numbers To Trap: 3
Actual Numbers To Seed: 1 2 3 4 5 6 7 19
Running 4 phase permute...


Game 1: 1 2 3 4 19
Game 2: 1 2 3 4 5
Game 3: 1 2 3 4 6
Game 4: 1 2 3 4 7
Game 5: 1 2 3 5 19
Game 6: 1 2 3 5 6
Game 7: 1 2 3 5 7
Game 8: 1 2 3 6 19
Game 9: 1 2 3 6 7
Game 10: 1 2 3 7 19
Game 11: 1 2 4 5 19
Game 12: 1 2 4 5 6
Game 13: 1 2 4 5 7
Game 14: 1 2 4 6 19
Game 15: 1 2 4 6 7
Game 16: 1 2 4 7 19
Game 17: 1 2 5 6 19
Game 18: 1 2 5 6 7
Game 19: 1 2 5 7 19
Game 20: 1 2 6 7 19

Output saved in file: output.1_2_3_4_5_6_7_19.091508.6504


Again, this script isn't meant to used for gambling purposes, although it could be. We only chose this scenario as so many facets of our state's lottery relate directly to the concepts we've been going over for the past 2 weeks and it made it fun for the author :)

In all seriousness, if you have a gambling problem (cards, roulette, lottery, whatever) call 1-800-GAM-BLER or visit 800gambler.org and get some help. Like pretty much everything, the lottery is only fun if it's not hurting you! And, oh yes, if it helps, this number picker does not increase the likelihood that any of the numbers it picks will match any random drawing. Luck, and however many other digits are involved in the number pool, make this a losing proposition, at best. To prove this, I ran a test against the Illinois Lottery's past-drawing numbers for 2008 and found that, for my modest investment of $5,040, I would have made $764 in return. In other words, unless you're lucky, this number selection system will only guarantee that you'll lose more than you gain. Like all the disclaimers all say; it's for entertainment purposes only!

Okay, we've done our part. The PSA is over. Enjoy the script (it, and its parts, can be useful for a variety of other purposes) :)

NOTE: The improved permutation subroutine was found online in an excerpt from the Perl Cookbook, by Tom Christiansen and Nathan Torkington - an excellent read :)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/usr/bin/perl

#
# wheel.pl
# TODO: out of order numseeds do not make correct output
#
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

print "Highest Number: ";
chomp($highest_number=<STDIN>);

print "Number Of Balls Drawn: ";
chomp($num_balls=<STDIN>);
$array_num_balls=$num_balls - 1;

print "How Many Numbers To Trap: ";
chomp($num_trap=<STDIN>);
$array_num_trap=$num_trap - 1;

print "Enter your numbers\n";
print "separated by space character : ";
chomp($num_seeds=<STDIN>);
$num_seeds =~ s/ /,/g;
@number_seeds = split(/,/, $num_seeds);
$number_seeds = @number_seeds;

if ( $number_balls > $num_seeds ) {
print "--------------------------\n";
print "Your picks $num_seeds\n";
print "is larger than $num_balls nums picked\n";
print "Try again!\n";
print "--------------------------\n";
exit(1);
}
foreach $unique_num (@number_seeds) {
if ($seen{$unique_num}++) {
print "--------------------------\n";
print "Your Number set $num_seeds\n";
print "contains duplicate numbers! Try again!\n";
print "--------------------------\n";
exit(2);
} elsif ( $unique_num > $highest_number ) {
print "--------------------------\n";
print "Your Number set $num_seeds contains\n";
print "a number $unique_num greater than\n";
print "your highest number value: ${highest_number}. Try again!\n";
print "--------------------------\n";
exit(3);
}
}

print "\n\n";
print "Highest Number $highest_number\n";
print "Balls Drawn $num_balls\n";
print "How Many Numbers To Trap: $num_trap\n";
print "Actual Numbers To Seed: @number_seeds\n";

$|=1;

@sperms = permoot(@number_seeds);
print "Running 4 phase permute...\n";

$useless_counter=0;
foreach $unordered (@sperms) {
$unordered =~ s/\s/,/g;
@unsorted = split(/,/, $unordered);
@crazy = @unsorted[0 .. $array_num_balls];
$unsorted = join(" ", @crazy);
@temparray = split(" ", $unsorted);
@temparray = sort { $a <=> $b } (@temparray);
unshift(@sorted_perms, "@temparray");
}

$unik="0";
$last_unik="0";
$useless_counter=0;
foreach $s_p (@sorted_perms) {
@unik = split(" ", $s_p);
@unik = sort(@unik);
$seen2{$s_p}++;
if ( $seen2{$s_p} >= 2 ) {
next;
}
if ( $unik == $last_unik ) {
unshift(@ball_draw_pool,"$s_p");
@last_unik = @unik;
$last_unik++;
} elsif ( @unik == @last_unik ) {
$total = @unik;
$zero = $nope = 0;
while ( $zero < $total ) {
if ( "$unik[$zero]" ne "$last_unik[$zero]" ) {
$nope = 1;
last;
} else {
$zero++;
}
}
if ( $nope ) {
unshift(@ball_draw_pool,"$s_p");
@last_unik = @unik;
}
}
}


$useless_counter=0;
foreach $unordered (@sperms) {
$unordered =~ s/\s/,/g;
@unsorted = split(/,/, $unordered);
@crazy = @unsorted[0 .. $array_num_trap];
$unsorted = join(" ", @crazy);
@temparray = split(" ", $unsorted);
@temparray = sort { $a <=> $b } (@temparray);
unshift(@sorted_perms, "@temparray");
}

$useless_counter=0;
$unik="0";
$last_unik="0";
foreach $s_p (@sorted_perms) {
@unik = split(" ", $s_p);
@unik = sort(@unik);
$seen2{$s_p}++;
if ( $seen2{$s_p} >= 2 ) {
next;
}
if ( $unik == $last_unik ) {
unshift(@ball_trap_pool,"$s_p");
@last_unik = @unik;
$last_unik++;
} elsif ( @unik == @last_unik ) {
$total = @unik;
$zero = $nope = 0;
while ( $zero < $total ) {
if ( "$unik[$zero]" ne "$last_unik[$zero]" ) {
$nope = 1;
last;
} else {
$zero++;
}
}
if ( $nope ) {
unshift(@ball_trap_pool,"$s_p");
@last_unik = @unik;
}
}
}
print "\n";

foreach $smaller (@ball_trap_pool) {
@smaller = split(" ",$smaller);
foreach $larger (@ball_draw_pool) {
if ( $larger =~ $smaller[0] && $larger =~ $smaller[1] && $larger =~ $smaller[2] ) {
$total_pool{$larger} .= $smaller;
last;
$|=1;
}
}
}

chomp($date=`date +%m%d%y`);
$var = $$;
$con_num_seeds = $num_seeds;
$con_num_seeds =~ s/,/_/g;
$lfile_name = "output.${con_num_seeds}.${date}.${var}";
print "\n";
open(LFILE, ">$lfile_name") or die "Cannot open FILE";
print LFILE "Output File output.${con_num_seeds}.${var}\n";
print LFILE "--------------------------------\n";
print LFILE "$highest_number Ball Selection - Pick $num_balls - Trap $num_trap\n";
print LFILE "Seed Numbers: $number_seeds\n";
print LFILE "--------------------------------\n";

$counter = 1;
foreach $key (sort keys %total_pool ) {
print "Game $counter: $key\n";
$|=1;
print LFILE "Game $counter: $key\n";
$counter++;
}
close(LFILE);
print "\nOutput saved in file: $lfile_name\n";

sub permoot {
#
# The improved permutation subroutine was found online in an excerpt from the Perl Cookbook, by Tom Christiansen and Nathan Torkington
@DATA = @_;
foreach ("@DATA") {
my @data = split;
my $num_permutations = factorial(scalar @data);
for (my $i=0; $i < $num_permutations; $i++) {
@permutation = @data[n2perm($i, $#data)];
@sortperms = join(" ", @permutation);
push(@sperms, "@sortperms" );
}
}
return @sperms;

BEGIN {
my @fact = (1);
sub factorial($) {
my $n = shift;
return $fact[$n] if defined $fact[$n];
$fact[$n] = $n * factorial($n - 1);
}
}

sub n2pat {
my $i = 1;
my $N = shift;
my $len = shift;
my @pat;
while ($i <= $len + 1) {
push @pat, $N % $i;
$N = int($N/$i);
$i++;
}
return @pat;
}

sub pat2perm {
my @pat = @_;
my @source = (0 .. $#pat);
my @perm;
push @perm, splice(@source, (pop @pat), 1) while @pat;
return @perm;
}

sub n2perm {
pat2perm(n2pat(@_));
}
}


, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Friday, September 12, 2008

Maximizing Set Match Probability Using Perl On Linux Or Unix

Hey There,

As promised (I think ;), we're back today with some code (not the entire script that encompasses the last 2 weeks worth of posts, of course) to grind out the final concept in this string of surprisingly wide-ranging topics all centered around the concepts of number pools and guaranteed matches. We've covered a lot of different aspects of mathematics, regular expression matching, etc just moving toward our Objective. So, aside from the link to our initial introductory number pools post, we'll just suffice it so say that every post since then (excluding the weekend humour) has been directly related to this mini-series' stated Objective. I'd love to label this paragraph with a whole slew of hyperlinks, but I'm going to pretend you're like me and don't want to get zapped to some other page every 5 seconds just because your finger twitches while you're reading this column ;) You should be able to access any of the older posts via the side menu or by just clicking on the "older posts" link at the bottom of the post itself.

BTW, thanks to an anonymous reader who pointed out that I accidentally goofed when I was manually typing out the distribution of number sets in yesterday's post. The line, in the first grid, that read:

12478 124 127 128 147 148 178 247 248 258 358
should have read:

12478 124 127 128 147 148 178 247 248 278 478

I got sloppy while I was doing the manual typing of the 3-digit strings and accidentally re-typed the last 2 3-digit sets from the previous line. Hopefully, I've fixed it by the time you read this ;)

As I worked out this last part (the brain teaser being directly related here), I found that, although I could squeeze all 3-digit combinations of our 8 numbers into 8 5-digit lists of numbers, the ratio of match to payoff was fairly low (depends on what you're looking for, I suppose) as laid out in the statistics below:

FORMAT: List of 8 5-Digit Sets

12346 12358 12478 13678 14567 23457 25678 34568

FORMAT: Approx % of set pool == # of times matched == # of sets == list of 3-digit matches
41% == 2 == 23 == 123 124 128 136 138 146 147 167 178 234 235 247 257 258 278 345 346 358 368 456 457 567 568
59% == 1 == 33 == 125 126 127 134 135 137 145 148 156 157 158 168 236 237 238 245 246 248 256 267 268 347 348 356 357 367 378 458 467 468 478 578 678


Of course, the idea here is to get the maximum amount of unique matches in a 5-digit set, but it would be nice to have more than a single or double return on each hit (and a greater weighted percentage on the 3-digit sets that match more than once in different lines on the grid), so that the statistical difference is more in your favour (Note: This post is not about the Lottery. We neither recommend nor discourage you from gambling. It's not our place. If you must gamble, only bet what you can afford to lose. That's just good advice all around ;) This is pretty bad, with a 41%/59% ratio of above-single-match-hits vs. single-match-hits

If we took the following Perl code (runnable on any Linux or Unix distro that supports Perl 5.6 and up -- the behaviour of hashes changed slightly around the 5.6 release), which iterates through the 56 5-digit lists we started getting into on our posts on sorting Perl lists and finding the maximum number of pool sets using binomial coefficient theory. The 5-digit list while-loop is wrapped in another while-loop that iterates through all the possible 3-digit sets (since we want to match 3 digits minimum) and populates a hash with keys and values to determine what lists (in our 56 member 5-digit set list) are viable for matching all possible 3-digit matches and returning the highest probable yield. You'll notice that the output is a little skewed toward the lower numbers due to the matching algorithm:

foreach $smaller (@tri_set_pool) {
@smaller = split("",$smaller);
foreach $larger (@five_digit_pool) {
if ( $larger =~ $smaller[0] && $larger =~ $smaller[1] && $larger
=~ $smaller[2] ) {
$total_pool{$larger} .= $smaller;
last;
$|=1;
}
}
}
$counter = 1;
foreach $key (sort keys %total_pool ) {
print "Game $counter: $key\n";
$counter++;
}


Note that this code would give us "20" 5-digit match strings (up from our original 8), but a disproportionately better chance of a larger return (closer to the beginning, as noted above, which could be counted as a flaw ;)

FORMAT: List of 20 5-Digit Sets
12345 12346 12347 12348 12356 12357 12358 12367 12368 12378 12456 12457 12458 12467 12468 12478 12567 12568 12578 12678

FORMAT: Approx % of set pool == # of times matched == # of sets == list of 3-digit matches
1% == 11 == 1 == 125
6% == 10 == 3 == 123 124 128
1% == 8 == 1 == 127
1 % == 5 == 1 == 126
50% == 4 == 28 == 134 135 136 137 138 145 146 147 148 156 157 158 178 234 235 236 237 238 245 246 247 248 256 257 258 267 268 278
4% == 3 == 2 == 167 168
37% == 1 == 20 == 345 346 347 348 356 357 358 367 368 378 456 457 458 467 468 478 567 568 578 678


So, that's obviously a much better distribution of multiple 3-digit match occurrences (i.e., if you matched 123, you'd match it 10 times!) This tips thing back to good with a 63%/37% ratio of above-single-match-hits vs. single-match-hits.

Finally, one simple addition of a reversed 5-digit list array in the code, attacking the 3-digit matches against both and stripping off all 5-digit numbers that only contain a single match (note that this does not mean that no single match occurrences were found for any given 3-digit list!), gives us the best (as in most even) odds so far, although possibly not the best return on a win (statistics below the additional code following - pardon the cheap editor tricks ;) - This is another 20 list return, but with favorable-match odds that make it worth going over the modest 8-list list we started out with. Note, also, that this revision takes out the humungous skew between beginning and end (from 11 matches to 1 match) that our last revision created:

@five_digit_pool = qw(12345 12346 12347 12348 12356 12357 12358 12367 12368 12378 12456 12457 12458 12467 12468 12478 12567 12568 12578 12678 13456 13457 13458 13467 13468 13478 13567 13568 13578 13678 14567 14568 14578 14678 15678 23456 23457 23458 23467 23468 23478 23567 23568 23578 23678 24567 24568 24578 24678 25678 34567 34568 34578 34678 35678 45678);
@rev_five_digit_pool = qw(45678 35678 34678 34578 34568 34567 25678 24678 24578 24568 24567 23678 23578 23568 23567 23478 23468 23467 23458 23457 23456 15678 14678 14578 14568 14567 13678 13578 13568 13567 13478 13468 13467 13458 13457 13456 12678 12578 12568 12567 12478 12468 12467 12458 12457 12456 12378 12368 12367 12358 12357 12356 12348 12347 12346 12345);
@tri_set_pool = qw(123 124 125 126 127 128 134 135 136 137 138 145 146 147 148 156 157 158 167 168 178 234 235 236 237 238 245 246 247 248 256 257 258 267 268 278 345 346 347 348 356 357 358 367 368 378 456 457 458 467 468 478 567 568 578 678);

foreach $smaller (@tri_set_pool) {
@smaller = split("",$smaller);
foreach $larger (@five_digit_pool) {
print "$larger =~ $smaller[0] && $larger =~ $smaller[1] && $larger =~ $smaller[2]\n";
if ( $larger =~ $smaller[0] && $larger =~ $smaller[1] && $larger =~ $smaller[2] ) {
$total_pool{$larger} .= $smaller;
last;
$|=1;
}
}
}
foreach $smaller (@tri_set_pool) {
@smaller = split("",$smaller);
foreach $larger (@rev_five_digit_pool) {
print "$larger =~ $smaller[0] && $larger =~ $smaller[1] && $larger =~ $smaller[2]\n";
if ( $larger =~ $smaller[0] && $larger =~ $smaller[1] && $larger =~ $smaller[2] ) {
$total_pool{$larger} .= $smaller;
last;
$|=1;
}
}
}
$counter = 1;
foreach $key (sort keys %total_pool ) {
$total_poolkey = length $total_pool{$key};
if ( $total_poolkey <= 6 ) {
next;
}
print "$counter: $key -- $total_pool{$key}\n";
$counter++;
}


FORMAT: List of 20 5-Digit Sets
12345 12346 12347 12348 12356 12357 12358 12367 12368 12378 12678 13678 14678 15678 23678 24678 25678 34678 35678 45678

FORMAT: Approx % of set pool == # of times matched == # of sets == list of 3-digit matches
64% == 2 == 36 == 123 126 127 137 138 146 147 148 156 157 158 167 168 178 236 237 238 246 247 248 256 257 258 267 268 278 346 347 348 356 357 358 367 368 378 678
36% == 1 == 20 == 124 125 128 134 135 136 145 234 235 245 345 456 457 458 467 468 478 567 568 578


So, with this 64%/36% ratio of above-single-match-hits vs. single-match-hits, we've got about the same statistical odds as our last rendition. This version should get you above a single-match more often (since the other leaves you in single-match territory on all sets starting with 3 and up), but it does dampen your possible return (you'll never hit a 3-digit match that comes up more than twice).

Depending upon how you're using this sort of probabilistic determination, one version may appeal more to you than the other (And, again, if you use this to spread lottery picks, you still stand a really good chance of not having any of the numbers in your set match what they pick out of the air. I look at the lottery as straight 50% odds. They'll either pick my numbers or they won't, no matter how "overdue" any combinations may be (Believe me, if you live in Chicago and follow the Cubs, you can witness the fallacy of the "overdue win" in real-life every year ;) It's all random and the future can't be inferred from past events ...unless your local lottery is crooked ;)

Anyway, I hope you all have a great weekend, and I'll be sure to pop the cleaned up (and all put together) version of the script, that encompasses the last 2 weeks worth of post topics, as soon as I can next week. Here's hoping I can find you some more funny stuff for the weekend. I promise: No number jokes ;)

Cheers,

, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Thursday, September 11, 2008

Evaluating Number Grids Using Perl On Linux Or Unix

Hey There,

And, we're back from an evening of teasing our brains. I was actually quite cruel to mine. Hopefully, it will forgive me ;)

Here's hoping everyone's having an enjoyable day at work (or, even better, an enjoyable drink on the beach :). I'm currently wading through a sea of bureaucracy the complexity of which demands to be taken in with nothing less than total awe. The term Byzantine been taken to another level of meaning for me. It's always an interesting thing in life when you realize that, just the other day, you thought you had it all figured out ;) But, on the flipside, managers and directors need to manage and direct, so I don't begrudge them their duties, even if they seem to be manufactured to a certain degree. I'll gladly fill out a few forms if it'll keep somebody's family in food and clothing with a roof over their heads. My nagging inner-voices are fickle, selfish, somewhat pedestrian and completely unrealistic. It's a good thing I realized this about myself so many years ago. Bless my mother and father for not giving up on my sorry tween-age behind. Having a few kids of my own, I realize ( a little more with each passing day ) why my parents kept me around and why, every once and a while, they wanted to kill me ;)

Anyway, this nightmare of form stamping, filing and dull aching headaches (Think Terry Gilliam's Brazil except not in the future and with nice bright halogen lighting to offset the bizarre floral-patterned non-threatening beige cubicle farms ;) hasn't given me any time to script out my answer to yesterday's brain teaser. Believe it or not, I actually did the original half on paper and then half in my head. Actually, you'll believe it. When you see it in print, it becomes pretty simple to see the pattern.

Hopefully, you didn't give the brain-teaser too much effort if the answer didn't appear to you quickly. Like I said, I wasted a lot of time looking for suitable number sequences ( golden strings, etc ), but, in the end, found that I could reasonably deduce the most economical 8 out of 56 lists by just looking at them. Admittedly, it's a lot easier to read now that I've grouped them together (looking at all 56 was alternately confusing and just bad for my eyesight in general ;) Actually, a lot of you can probably knock out a quick Perl or shell script for Linux or Unix to deal with the pattern matching and list collecting just from seeing the grid of lists in one place.

The simple answer to the question (without the equation part, which I promise I'll put out as an aside to a post on something else original so you won't have to hear about numbered lists any more after we post the script that takes the last 2 weeks of posts on number pools, probability, binomial theory and , of course, lists and automates them ...at least, until I get sick of stuff other than lists ;) is that these 8 lists combined the most dense and absolutely complete set of all 3 digit lists within my 56 5 digit lists. The list of all 3 digit lists is simple to chart out (even if our list wasn't just "1, 2, 3, 4, 5, 6, 7 ,8)" and is shown below:

123 124 125 126 127 128 134 135 136 137 138 145 146 147 148 156 157 158 167 168 178 234 235 236 237 238 245 246 247 248 256 257 258 267 268 278 345 346 347 348 356 357 358 367 368 378 456 457 458 467 468 478 567 568 578 678


We'll leave the group of 20 lists that make for the most economical group of lists for a 4 digit match alone (same principle), and take a look, again, at the 8 lists which most economically covered every single list within the 3 digit match list above (can I type list again? I think I can... ;) These 8 lists are listed in the format: five-digit-list (space) space-delimited listing of all 3 digit matches covered.

12346 123 124 126 134 136 146 234 236 246 346
12358 123 125 128 135 138 158 235 238 258 358
12478 124 127 128 147 148 178 247 248 278 478
13678 136 137 138 167 168 178 367 368 378 678
14567 145 146 147 156 157 167 456 457 467 567
23457 234 235 237 245 247 257 345 347 357 457
25678 256 257 258 267 268 278 567 568 578 678
34568 345 346 348 356 358 368 456 458 468 568


Now, compare that with the list of 8 random lists, taken from the same group of 56, below, in the same format.

12458 124 125 128 145 148 158 245 248 258 358
12457 124 125 127 145 147 157 245 247 257 457
12456 124 125 126 145 146 156 245 246 256 456
12378 123 127 128 137 138 178 237 238 278 378
45678 456 457 458 467 468 478 567 568 578 678
35678 356 357 358 367 368 378 567 568 578 678
34678 346 347 348 367 368 378 467 468 478 678
34578 345 347 348 357 358 378 457 458 478 578


One thing you'll notice, just by looking, is that every single 3 digit match can be found in the first 8 list set, but not all can be found in the second. As for the reason that my answers always come up identical, this may be a trick of circumstance caused by the numbers themselves. I started my parsing from the 1's up (for all 56 lists in numeric order) and found the 8 identified as being a nice tidy all-encompassing grid "first." And I quote first because I didn't bother to check and see if I could find another one afterward. It (and several others) may exist in the list of 56 lists, even after my conditions for completion are satisfied. In fact, by default, if you use all 56 lists, you're guaranteed the "absolute" best coverage of all 3 digit matches, because you'll be hitting every single one of them (all 56 unique variations of 3 digit lists using the numbers 1 through 8, with no repetition or duplication).

To see what I mean, just check for random 3 digit strings in both matrices (or lists of lists). For instance, see if you can find 134, 135 or 136 in both sets or if they only appear in the top one (the one that's supposed to cover all 56 variations).

Enjoy, and I'll crank out a little script to replicate what I have written on paper and stuck in my head for you in the next post or so. As long as you understand the purpose of the grids (covering all variations of the 56 3 digit lists) it becomes trivial for you to create your own complete lists of lists (from the 56 lists you can make of 5 digit lists spreading over 8 digits) and experience for yourself what it is to understand something and feel somewhat at a loss to adequately explain it ;)

Cheers,

, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Wednesday, September 10, 2008

Brain Teaser: Seemingly Random Number List Selection

Hey there,

We're just about done with this series of somewhat disjointed posts on number pools and guaranteed matches. Hopefully, each post in the series has been somewhat entertaining and far enough removed from the starting post that we kept up some sense of originality. It wasn't too hard (unless we screwed up ;) since there were so many unique concepts to cover within the realm of reaching the Objective of the original post. Please refer back to our post on number pools for definitions of all the major terms and a listing of the Objective (although that definition is repeated in a few other posts along the way).

We're just about at the end of the road, so I thought I'd make today's post a bit of a Brain Teaser. The answer to this is actually pretty interesting and (from my struggling with it) you'll either get it right away or it'll take you a good amount of time (kind of like the M I U Puzzle). I, for one, spent way too much time reading up on number sequences and probability. I'm not an engineering major, so I actually had to reference other materials to decipher the Greek alphabet notation. It was interesting for me (especially the insomniac in me), but not something I'd recommend doing unless you're really into it. One thing I noticed, although this is slightly off topic; when people see you sitting in your easy chair chugging diet Pepsi, popping Diphenhydramine and typing in your terminal window with your lap covered with shreds of paper with interpretive numeric scrawl written all over them, they tend to not ask too many questions. It's a great way to guarantee you'll have the evening to yourself ;)

Anyway; for this post's meat: The brain teaser we have today is pretty straightforward, by way of explanation. Here's the setup:

1. Like we did in our posts on deriving list permutations, sorting lists and sorting lists within lists (not necessarily in that order), we've generated a list of 56 5 digit lists using an 8 digit pool of numbers (1, 2, 3, 4, 5, 6, 7, 8). This list represents all possible permutations of the 8 digit list within 5 digit lists while "absolutely removing duplicates" (i.e. "1, 2, 3, 4, 5" and " 5, 3, 2, 4, 1" would be considered equal and would both be removed). That list of numbers follows here (Note that each line is numbered):

1 1 2 3 4 5
2 1 2 3 4 6
3 1 2 3 4 7
4 1 2 3 4 8
5 1 2 3 5 6
6 1 2 3 5 7
7 1 2 3 5 8
8 1 2 3 6 7
9 1 2 3 6 8
10 1 2 3 7 8
11 1 2 4 5 6
12 1 2 4 5 7
13 1 2 4 5 8
14 1 2 4 6 7
15 1 2 4 6 8
16 1 2 4 7 8
17 1 2 5 6 7
18 1 2 5 6 8
19 1 2 5 7 8
20 1 2 6 7 8
21 1 3 4 5 6
22 1 3 4 5 7
23 1 3 4 5 8
24 1 3 4 6 7
25 1 3 4 6 8
26 1 3 4 7 8
27 1 3 5 6 7
28 1 3 5 6 8
29 1 3 5 7 8
30 1 3 6 7 8
31 1 4 5 6 7
32 1 4 5 6 8
33 1 4 5 7 8
34 1 4 6 7 8
35 1 5 6 7 8
36 2 3 4 5 6
37 2 3 4 5 7
38 2 3 4 5 8
39 2 3 4 6 7
40 2 3 4 6 8
41 2 3 4 7 8
42 2 3 5 6 7
43 2 3 5 6 8
44 2 3 5 7 8
45 2 3 6 7 8
46 2 4 5 6 7
47 2 4 5 6 8
48 2 4 5 7 8
49 2 4 6 7 8
50 2 5 6 7 8
51 3 4 5 6 7
52 3 4 5 6 8
53 3 4 5 7 8
54 3 4 6 7 8
55 3 5 6 7 8
56 4 5 6 7 8


2. Using the technique we looked at in our post on determining maximum sets of lists within pools using binomial coefficients, we've deduced that, since we want to guarantee (within the lists in this list of lists) unique 3 digit combinations, we can do so most economically by selecting 8 of the 5 digit lists from the 56. This is not the exact formula we followed in the noted post (as it showed that the binomial coefficient of 8 over 3 is 56). In fairly loose terms, if we want to maximize the odds that we'll get any given 3 digit list (and not use all 56 lists, which works, but is expensive), we can do so with a minimum of 8 5 digit lists. Those 8 lists follow here:

1 2 3 4 6
1 2 3 5 8
1 2 4 7 8
1 3 6 7 8
1 4 5 6 7
2 3 4 5 7
2 5 6 7 8
3 4 5 6 8


3. Using the numbers from point 1 and 2, above, the following chart shows you at what line (in the list of 5 digit lists) each of the 8 lists we determined would give us the best odds of getting a maximum return on a 3 digit list match up.

2 1 2 3 4 6 1 2 3 4 6
7 1 2 3 5 8 1 2 3 5 8
16 1 2 4 7 8 1 2 4 7 8
30 1 3 6 7 8 1 3 6 7 8
31 1 4 5 6 7 1 4 5 6 7
37 2 3 4 5 7 2 3 4 5 7
50 2 5 6 7 8 2 5 6 7 8
52 3 4 5 6 8 3 4 5 6 8


4. Using the numbers from point 1 and 2, above, the following chart shows you at what line (in the list of 5 digit lists) each of the 20 lists we can also determine would give us the best odds of getting a maximum return on a 4 digit list match (We never covered this, but I just added it to give you some frame of reference if the above 3 points aren't enough - BTW, I generated this list because they weren't enough for me, so no offense intended :)

1 1 2 3 4 5 1 2 3 4 5
8 1 2 3 6 7 1 2 3 6 7
9 1 2 3 6 8 1 2 3 6 8
14 1 2 4 6 7 1 2 4 6 7
15 1 2 4 6 8 1 2 4 6 8
18 1 2 5 6 8 1 2 5 6 8
19 1 2 5 7 8 1 2 5 7 8
21 1 3 4 5 6 1 3 4 5 6
22 1 3 4 5 7 1 3 4 5 7
23 1 3 4 5 8 1 3 4 5 8
26 1 3 4 7 8 1 3 4 7 8
35 1 5 6 7 8 1 5 6 7 8
38 2 3 4 5 8 2 3 4 5 8
39 2 3 4 6 7 2 3 4 6 7
42 2 3 5 6 7 2 3 5 6 7
45 2 3 6 7 8 2 3 6 7 8
46 2 4 5 6 7 2 4 5 6 7
49 2 4 6 7 8 2 4 6 7 8
52 3 4 5 6 8 3 4 5 6 8
53 3 4 5 7 8 3 4 5 7 8


Now, here's the fun part. Considering that:

a) no matter how many times we do this using the exact parameters, the results (the lists and the placement of the 8 most economical choices) are totally consistent.
b) If we change anything (like make our list of "1, 2, 3, 4, 5, 6, 7, 8" into "1, 4, 3, 2, 5, 6, 7, 8") about our 8 digit list (assuming that we don't resort everything to numeric order before generating our 8 best 5 digit lists), the outcome will be affected in that the 8 lists will not only contain different numerals, but also end up matching at different numbered locations on master list of 56 lists.

Considering that, how am I determining where to place my 8 5 digit lists in relation to my 56 list list. Or, in other terms, how am I picking the 8 5 digit lists I feel will give me the best odds of matching a 3 digit combination of any 3 unique digits from the 8 digit list used to seed these 5 digit selections?

Think about it, but not too hard. This is only fun if you're having fun with it. If it starts to upset you, watch something violent, yet satisfying (like a good revenge flick), on television and get some sleep ;)

Cheers,

, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Thursday, September 4, 2008

Sorting Perl Lists And Removing Duplicates On Linux Or Unix

Hey there,

This may be the first time I've run a series of posts where the topic was different every step of the way. That's a good thing, because I get just as bored as anyone else reading about the same thing over and over ;) So, following in the footsteps of our posts on Number Pools And Guaranteed Combinations Within Fixed Lists and Perl List Permutations, we'll move straight ahead to the next logical step, which is almost an entirely different subject altogether. I will however, reiterate (repeat again... sorry, one of my idiosyncrasies ;) our Objective in this whole exercise (Skip to the next paragraph if you feel I've done this sentence to death ;) : Given a Number Pool of "x through y," create the maximum possible Fixed List Length variations of our Fixed List that contain some variation of our Guaranteed Combination, without any duplication (i.e. 1, 2, 3 is equal to 2, 3, 1 and would only count as one match), and return the results.

Of course, to revisit the definitions of our key components , please refer back to our previous post on Number Pools

Today, we're going to look a Perl list sorting. In this context, you can actually consider our lists as arrays if you prefer. I'm going to keep calling them lists because, in the end, it will most closely approximate what we will want to manipulate; a fixed list of numbers.

From yesterday's posts on list permutations you can basically understand, even if you didn't run the code, how very many different combinations of 6 numbers there can be, when duplication is not a factor (i.e. 1, 2, 3 is different than 3, 2, 1). In our case, given our Objective, we would want those number lists, in parentheses above, to be considered equal. So how do we go about that?

There are actually quite a number of ways you can go about doing this. They run the gamut from human-readable and computer-intensive to nearly-hieroglyphic and incredibly efficient. Some folks like to dump all their values into hashes and sort it out that way, since the Perl hash function is built to sort keys and values. I'm not against that at all, but feel like it might stray us farther off the path and I want to keep this series of posts somewhat accessible.

My answer is actually a fairly commonly used one, as well. I don't consider myself a Perl expert (or really even a Unix or Linux expert. Like in Eastern philosophy: Even the teacher is ever the student). If I ever do convince myself that I know it all, I'll probably have a fatal seizure from lack-of-interest within moments of my revelation ;)

I like to sort all of my permutations (in this case numerically) first, and then check for duplicates. This makes sense to me, for some reason, and (especially in this case) it can greatly reduce the pressure you feel when dealing with large numbers of numbers. For instance, our 720 permutations are all of the same 6 numbers, so, once they've all been sorted numerically, we're going to have 720 identical lists. We don't even need a computer to tell us what our Objective obliquely defines as a "unique list" in this situation. It's "1, 2, 3, 4, 5."

All of our actions are predicated on the output of the "public domain" code we put out yesterday, which has been revised slightly to suit our purposes (the original just printed out the answers and didn't do any variable assignation or a return) and, yes, I do use my lists as arrays from time to time (we all need time to relax and unshift ;) The "permute" subroutine is called with an array, or list, (@number_seeds) as its only argument (which contains "1, 2, 3, 4, 5"), like:

@s_perms = permute([@number_seeds], []);

Modified code below:

sub permute {
my @items = @{ $_[0] };
my @perms = @{ $_[1] };
unless (@items) {
# LIMITED TO ONE CONTIGUOUS PERMUTATION CHAIN
@sortperms = join(" ", @perms);
unshift(@s_perms, "@sortperms" );
} else {
my(@newitems,@newperms,$i);
foreach $i (0 .. $#items) {
@newitems = @items;
@newperms = @perms;
unshift(@newperms, splice(@newitems, $i, 1));
permute([@newitems], [@newperms]);
}
}
return @s_perms;
}


And that's the lesson for today. I've attached code to the end of the post to illustrate the steps of sorting each list in our list of lists and checking for (and removing) duplicate lists from the pool. Note that we don't take into account multiple variable lists, as this won't be germane later (and adds some heft to the scriplet :) I tried to make this as easy to read as possible, so various steps, which could be combined, have been spelled out. If it has the opposite effect and makes things more confusing, I apologize profusely. And let me know! If there's interest, I'd be happy to write about this stuff at a much more granular level :)

Here's the mock output, to save you a screenshot:

host # ./program
1 2 3 4 5


Enjoy!


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/usr/bin/perl

#
# program - an unoriginal name for an unoriginal concept ;)
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

foreach $unsorted (@s_perms) {
@temparray = split(" ", $unsorted);
@temparray = sort { $a <=> $b } (@temparray);
unshift(@sorted_perms, "@temparray");
}
#@sorted_perms = sort { $a <=> $b } (@s_perms);

$unik="0";
$last_unik="0";
foreach $s_p (@sorted_perms) {
@unik = split(" ", $s_p);
@unik = sort(@unik);
if ( $unik == $last_unik ) {
print "$s_p";
@last_unik = @unik;
$last_unik++;
} elsif ( @unik == @last_unik ) {
$total = @unik;
$zero = $nope = 0;
while ( $zero < $total ) {
if ( "$unik[$zero]" ne "$last_unik[$zero]" ) {
$nope = 1;
last;
} else {
$zero++;
}
}
if ( $nope ) {
print "$s_p";
@last_unik = @unik;
}
} else {
print "$s_p";
@last_unik = @unik;
}
}


, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Wednesday, September 3, 2008

Lists vs. Strings: Perl List Permutations For Linux Or Unix

Hey there,

I think I promised yesterday that all posts having to deal with Number Pools And Guaranteed Combinations Within Fixed Lists would have that string in their title, but, upon further reflection, it seems like it would make every title waaaay too long, and none of the parts would get indexed correctly, since Google might discard them as duplicate content based on the title tag.

In any event, this post is another small step toward yesterday's objective, along with (hopefully) an understanding of the various processes and steps involved in getting there. These are, indeed, small chunks of information when taken separately, but that is by intent. I'm not trying to insult anyone's intelligence; just keep the playing field even for all concerned ;) If you need to review the definitions of our system, please refer back to yesterday's post on Number Pools and we'll iterate (not reiterate of course, because that would be repeating it again. I'm only repeating it this time... OK, enough with the wordy nit-picking ;)

Today, we're going to look at one of the basics of building a system which can produce our desired end result. That basic building block is known as the permutation of a list, as opposed to an array (or string). Given Perl's definitions, at their basic, a list is static and cannot be modified, while the opposite is true of an array. They look deceptively the same. Most people don't know the difference. I still may not have a fingernail grip on the exact definition and details of the difference, but it hasn't caused me too much heartache ;) And, of course, strings are generally considered scalar variables (single value).

Our objective has been modified (in form, not in nature) to accommodate this change (and I've updated the original post's definitions as well): Given a Number Pool of "x through y," create the maximum possible Fixed List Length variations of our Fixed List that contain some variation of our Guaranteed Combination, without any duplication (i.e. 1, 2, 3 is equal to 2, 3, 1 and would only count as one match), and return the results.

In any event, my sincerest apologies for using string when I meant list. I didn't get a whole lot of complaints about this, which means either most everyone got what I meant or a vast majority of readers believe that Internet democracy will sort me out one way or another (which it has). This is what happens when you write at family gatherings ;)

So, given our Fixed List from yesterday (the numbers that, together, add up to a total of "Fixed List Length" members), we would first be looking for all permutations of that List. The list being composed of "1, 2, 3, 4, 5"

Using the following "public domain" code (meaning I didn't write it, don't accept credit for it, love it, but don't know who to credit it to ;) we can generate all permutations of our Fixed List. This will be the first thing we want to do, because this step needs to be completed before we check for Guaranteed Combinations within those permutations!

#!/usr/bin/perl

permute([1,2,3,4,5], []);

sub permute {
my @items = @{ $_[0] };
my @perms = @{ $_[1] };
unless (@items) {
print "@perms\n";
} else {
my(@newitems,@newperms,$i);
foreach $i (0 .. $#items) {
@newitems = @items;
@newperms = @perms;
unshift(@newperms, splice(@newitems, $i, 1));
permute([@newitems], [@newperms]);
}
}
}


This will produce all permutations of the List "1, 2, 3, 4, 5," as such (cropped for space savings):

5 4 3 2 1
4 5 3 2 1
5 3 4 2 1
3 5 4 2 1
4 3 5 2 1
3 4 5 2 1
...


As you can see, given our very simple example (where everything is equal to 5 and all lists and combinations are 5 (or 1, 2, 3, 4, 5), we have a lot of repetition in our output, and part of our stated objective is to create this list "without any duplication (i.e. 1, 2, 3 is equal to 2, 3, 1...)..."

Obviously, although this piece of Perl code will provide us with every possible permutation of the list "1, 2, 3, 4, 5," according to our Objective's definition, they're all equal. As we noted yesterday, the result can only be one list of five numbers, given our standards.

Tomorrow, we'll look at a way to resolve this issue, using a minimal amount of code (and, of course, a lot more of my concentric-circular explanation ;) so that the permutations get whittled down to the one unique list (containing all 5 numbers in our list) and removing all the "technically" different ones, which are essentially all the same list printed out in different order.

Cheers,

, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Tuesday, September 2, 2008

Number Pools And Guaranteed Combinations Within Fixed Lists

Hey There,

It's been a while since we've taken any time to look at any formal systems or number systems (all the way back to The M I U Puzzle, so today we're going to kick off a quick beginner post that is, somewhat Linux and Unix independent(although we'll be scripting and using Linux/Unix tools to do this work for us in future posts). It's going to segue into something more complicated in a future post, but, for today, we're just going to lay the groundwork. In fact, the beginnings of this process will seem almost brain-dead simple. Which is okay, because they pretty much are ;)

Our topic for today, and for all the follow-ups, is going to be: Number Pools and Guaranteed Combinations within Fixed Strings.

We'll adhere to the following definitions (Followed by our Objective):

Number Pool: A pool of unique numbers of any given amount spanning any given start and end numeric parameters.

Fixed List: A string of numbers of no more than the total number of numbers in the Number Pool and not containing any duplicate numbers.

Fixed List Length: The number of numbers in our Fixed List.

Guaranteed Combination: A combination of unique numbers (restricted to the members of the Fixed List we've defined) with a minimum member count of 1 and a maximum member count of the total number of numbers in our Fixed String.

Guaranteed Combination Length: The number of numbers in our Guaranteed Combination.

Our Objective: Given a Number Pool of "x through y," create the maximum possible Fixed List Length variations of our Fixed List that contain some variation of our Guaranteed Combination, without any duplication (i.e. 1, 2, 3 is equal to 2, 3, 1 and would only count as one match), and return the results.

Hopefully, that didn't sound too convoluted or complicated. It's not. Here's what we'll do to prove that very quickly.

1. Our Number Pool for today will be defined as: 1-5 (i.e. 1, 2, 3, 4, 5).
2. Our Fixed List will be: 1, 2, 3, 4, 5
3. This gives us a Fixed List Length of 5.
4. Our Guaranteed Combination, will be: 1, 2, 3, 4, 5
5. This gives us a Guaranteed Combination Length of 5.

So, the question before us is: Given a Number Pool of "1, 2, 3, 4, 5," what are the maximum possible variations of the 5 digit Fixed List "1, 2, 3, 4, 5" that we can create which will contain our Guaranteed Combination, without any duplication? (and, remember, this post is just to get you started thinking about the theory we'll put into practice later)

Simple deduction will lead you to this answer, since there can be only one correct answer (Which, in a sense, is true of larger systems, as well) and our variables are equal. Our result would be 1 variation:

1, 2, 3, 4, 5 <-- Note that the list 5, 4, 2, 1, 3 would be equal and would also be a correct answer, but would not count as an additional variation since it is made up of numbers duplicated from the original variation of 1, 2, 3, 4, 5.

Think about this for a while. It's a good exercise for your head :) In future posts we'll be looking at different variable expansions and the number theory behind generating these sort of lists (For example, a Number Pool of "1 through 99" with a Fixed String of "1, 3, 4, 6, 8, 9, 13, 17, 18, 20, 33, 47, 56, 68, 77, 87, 93" and a Guaranteed Combination of "1, 4, 6, 47, 68" - There will be a lot more strings to produce given these circumstances, which is where Linux or Unix scripting, as mentioned above, will help tremendously). Of course, we'll attack each individual part separately before combining them all into one equation (which will actually be somewhat simpler, but I guess that's a matter of opinion ;)

Cheers,

, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Sunday, April 27, 2008

Perl Script To Translate Signal Names And Numbers

Hey There,

For this "Lazy Sunday" post I threw together a quick little Perl script to help out further with what we went over more thoroughly in our previous post on signal definitions in Linux and Unix.

This script takes at least one argument (or it throws a fit and refuses to cooperate with you ;) of either a signal name or signal number, which it will then translate to the other (does the way I wrote that make sense? ;)

So, if you want to know what signals 4 and 5 do, you could run:

host # ./siggy.pl 4 5
Signal Number 4 = ILL
Signal Number 5 = TRAP


And see the (somewhat more verbose) translations from the system includes. Conversely, you could want to know what numbers the TERM and KILL signals are, which you could get by running:

host # ./siggy.pl TERM KILL
Signal Name TERM = 15
Signal Name KILL = 9


Or you can mix and match. Do whatever, you want. If you throw the script an argument it doesn't understand it will react accordingly ;)

host # ./siggy.pl hup 8 ... 9
Signal Name HUP = 1
Signal Number 8 = FPE
...: What is that supposed to mean?
Signal Number 9 = KILL


One interesting thing in the script is the use of an absolute subroutine call. Even though I put the "use Sys::SigAction;" line in the script, Perl was interpreting the function calls (like sig_number()) as belonging to the Main:: module. Pretty much every function, or command, in Perl does. "print," for example, is actually Main::print, so I had to use "absolute" naming. Writing "Sys::SigAction::sig_number()" (in this instance) is one way to make sure that Perl knows exactly where to look for that subroutine to run :)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/usr/bin/perl

#
# siggy.pl
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

if ( $#ARGV < 0 ) {
print "Usage: $0 signalNumber1 [signalNumbern...]\n";
exit(1);
}

use Sys::SigAction;

@signos = @ARGV;

foreach $sig (@signos) {
if ( $sig =~ /[A-Za-z]+/ ) {
$sig = uc($sig);
$signum=Sys::SigAction::sig_number( $sig );
print "Signal Name $sig = $signum\n";
} elsif ($sig =~ /[0-9]+/) {
$signame=Sys::SigAction::sig_name( $sig );
print "Signal Number $sig = $signame\n";
} else {
print "$sig: What is that supposed to mean?\n";
}
}


, Mike

Tuesday, November 6, 2007

Weird Obsession With Fibonacci Numbers - Brute Force Scripting!

Every once in a while, I cruise shell, perl and other unix/linux forums and try to help people out by answering questions and, inevitably, learning more than I impart.

The reason I mention this is the reason for today's post and it's strange title. For some reason, lately, lots of folks have been looking for code to list out Fibonacci numbers up to the highest number equal to or lesser than the one given on the command line as an argument. Of course, I smell a homework assignment here, but, why not just do it anyway?

Of course, even though I'd heard of the numbering sequence and formula, I looked at Wikipedia's Definition to make sure I remembered the sequencing correctly.

Then I thought I'd write a script as fast as I could to complete this task. It reminded me again of another very interesting principle. You can get a lot done in a very short period of time by being totally inefficient. It's especially o.k. if it's not your homework that you're doing ;)

For example, take this script that I whipped up in a few minutes. It takes an argument of one number (the outer delimiter) and it will print out all the Fibonacci numbers up to that argument value, unless that number value isn't a Fibonacci number, in which case it will print up to the last Fibonacci number less than that argument.


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

The incredibly lame script:

#!/bin/ksh

#
# 2007 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

final_number=$1
if [ -z $final_number ]
then
echo "No delimiter given. Exiting"
exit 1
fi
if [ $final_number -eq 0 ]
then
print "0"
exit 0
elif [ $final_number -eq 1 ]
then
print "0 1 1"
exit 0
else
print -n "0 "
first_number=1
fib=1
fi
start=1
while [ $fib -lt $final_number ]
do
if [ $start -eq 1 ]
then
print -n "1 "
last_number=0
next_number=1
start=0
fi
let fib=$last_number+$next_number
last_number=$next_number
next_number=$fib
if [ $fib -le $final_number ]
then
print -n "$fib "
fi
done
echo
exit 0


sample output:

$ ./fibo.sh 100000
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711 28657 46368 75025


Less than a page of code and not much thought put into making an efficient script. I decided to call it "Brute Force Scripting." Hey; sometimes your time is better spent not worrying about making things look pretty. If you get the job done and it's your job to get the job done, getting the job done is job number one ;)

, Mike